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Foreword 


Spivak and Me 


The document you are currently reading began its life as the lecture notes for a year 
long undergraduate course in honors calculus. Specifically, during the 2011-2012 
academic year I taught Math 2400(H) and Math 2410(H), Calculus With Theory, 
at the University of Georgia. This is a course for unusually talented and motivated 
(mostly first year) undergraduate students. It has been offered for many years at 
the University of Georgia, and so far as I know the course text has always been 
Michael Spivak’s celebrated Calculus [S]._ The Spivak text’s take on calculus is 
sufficiently theoretical that, although it is much beloved by students and practi- 
tioners of mathematics, it is seldomed used nowadays as a course text. In fact, the 
UGA Math 2400/2410 course is traditionally something of an interpolation between 
standard freshman calculus and the fully theoretical approach of Spivak. My own 
take on the course was different: I treated it as being a sequel to, rather than an 
enriched revision of, freshman calculus. 


I began the course with a substantial familiarity with Spivak’s text. The sum- 
mer after my junior year of high school I interviewed at the University of Chicago 
and visited Paul Sally, then (and still now, as I write this in early 2013, though 
he is 80 years old) Director of Undergraduate Mathematics at the University of 
Chicago. After hearing that I had taken AP calculus and recently completed a 
summer course in multivariable calculus at Johns Hopkins, Sally led me to a sup- 
ply closet, rummaged through it, and came out with a beat up old copy of Spivak’s 
text. “This is how we do calculus around here,” he said, presenting it to me. During 
my senior year at high school I took more advanced math courses at LaSalle Uni- 
versity (which turned out, almost magically, to be located directly adjacent to my 
high school) but read through Spivak’s text. And I must have learned something 
from it, because by the time I went on to college — of course at the University of 
Chicago — I placed not into their Spivak calculus course, but the following course, 
“Honors Analysis in R””. This course has the reputation of sharing the honor with 
Harvard’s Math 55 of being the hardest undergraduate math course that American 
universities have to offer. I can’t speak to that, but it was certainly the hardest 
math course I ever took. There were three ten week quarters. The first quarter was 
primarily taught out of Rudin’s classic text [R], with an emphasis on metric spaces. 
The second quarter treated Lebesgue integration and some Fourier analysis, and 
the third quarter treated analysis on manifolds and Stokes’s theorem. 

However, that was not the end of my exposure to Spivak’s text. In my second 
year of college I was a grader for the first quarter of Spivak calculus, at the (even 
then) amazingly low rate of $20 per student for the entire 10 week quarter. Though 
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the material in Spivak’s text was at least a level below the trial by fire that I had 
successfully endured in the previous year, I found that there were many truly dif- 
ficult problems in Spivak’s text, including a non-negligible percentage that I still 
did not know how to solve. Grading for this course solidified my knowledge of this 
elementary but important material. It was also my first experience with reading 
proofs written by bright but very inexperienced authors: often I would stare at 
an entire page of text, one long paragraph, and eventually circle a single sentence 
which carried the entire content that the writer was trying to express. I only graded 
for one quarter after that, but I was a “drop in tutor” for my last three years of 
college, meaning that I would field questions from any undergraduate math course 
that a student was having trouble with, and I had many further interactions with 
Spivak’s text. (By the end of my undergraduate career there were a small number 
of double-starred problems of which I well knew to steer clear.) 


Here is what I remembered about Spivak’s text in fall of 2011: 
(i) It is an amazing trove of problems, some of which are truly difficult. 
(ii) The text itself is lively and idiosyncratic.1 


(iii) The organization is somewhat eccentric. In particular limits are not touched 
until Chapter 5. The text begins with a chapter “basic properties of numbers”, 
which are essentially the ordered field axioms, although not called such. Chapter 
3 is on Functions, and Chapter 4 is on Graphs. These chapters are essentially con- 
tentless. The text is broken up into five “parts” of which the third is Derivatives 
and Integrals. 


After teaching the 2400 course for a while, I lost no esteem for Spivak’s text, 
but increasingly I realized it was not the ideal accompaniment for my course. For 
one thing, I realized that my much more vivid memories of the problems than the 
text itself had some basis in fact: although Spivak writes with a lively and distinct 
voice and has many humorous turns of phrase, the text itself is rather spare. His 
words are (very) well chosen, but few. When one takes into account the ample 
margins (sometimes used for figures, but most often a white expanse) the chapters 
themselves are very short and core-minded. When given the chance to introduce a 
subtlety or ancillary topic, Spivak almost inevitably defers it to the problems. 


I have had many years to reflect on Spivak’s text, and I now think its best use 
is in fact the way I first encountered it myself: as a source of self study for bright, 
motivated students with little prior background and exposure to university level 
mathematics (in our day, this probably means high school students, but I could 
imagine a student underwhelmed by an ordinary freshman calculus class for which 
Spivak’s text would be a similarly mighty gift). Being “good for self study” is a 
high compliment to pay a text, and any such text can a fortiori also be used in a 
course...but not in a completely straightforward way. For my course, although the 


lin between the edition of the book that Sally had given me and the following edition, all 
instances of third person pronouns referring to mathematicians had been changed from “he” to 
“she”. Initially I found this amusing but silly. More recently I have begun doing so myself. 
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students who stuck with it were very motivated and hard-working, most (or all) of 
them needed a lot of help from me in all aspects of the course. I had been informed 
in advance by my colleague Ted Shifrin that signing on to teach this course should 
entail committing to about twice as many office hours as is typical for an under- 
graduate course, and this did indeed come to pass: I ended up having office hours 
four days a week, and I spent the majority of them helping the students make their 
way through the problems in Spivak’s text. Even the best students — who, by the 
way, I thought were awfully good — could not solve some of the problems unassisted. 
There were many beautiful, multi-part problems introducing extra material that I 
wanted my students to experience: and eventually I figured out that the best way 
to do this was to incorporate the problems into my lectures as theorems and proofs. 


All in all I ended up viewing Spivak’s book as being something of a “deconstruc- 
tion”? of the material, and much of my teaching time was spent “reconstructing” it. 


For me, the lightness of touch of Spivak’s approach was ultimately something I 
appreciated aesthetically but could see was causing my students difficulty at vari- 
ous key points. My experience rather convinced me that “more is more”: students 
wanted to see more arguments in complete detail, and also simply more proofs over- 
all. Aside from the (substantial) content conveyed in the course, a major goal is to 
give the students facility with reading, understanding, constructing and critiquing 
proofs. In this regard being in a classroom setting with other motivated students is 
certainly invaluable, and I tried to take advantage of this by insisting that students 
present their questions and arguments to each other as well as to me. 

But students also learn about how to reason and express themselves mathemat- 
ically by being repeatedly exposed to careful, complete proofs. I think experienced 
mathematicians can forget the extent to which excellent, even brilliant, students 
benefit from this exposure. Of course they also benefit immensely, probably more 
so, by working things out for themselves. The value of this is certainly not in 
question, and there is no competition here: the amount of “things to do” in honors 
calculus is infinite — and the repository of problems in Spivak’s text is nearly so — 
so by working out more for herself, the instructor is not leaving less for the students 
to do, but only different things for them to do. 


This brings me to the current text. As explained above, it is heavily indebted 
to [S]. However, it is — or at least, I mean it to be — a new honors calculus text, 
and not merely a gloss of [S]. Indeed: 


e The text is indebted to [S], but not uniquely so. It is also heavily indebted 
to Rudin’s classic text [R], and it borrows at key points from several other sources, 


e.g. [A], [Go], [A], [1]. 


I do not view this borrowing as being in any way detrimental, and I certainly do 
not attempt to suppress my reliance on other texts. The mathematics exposed here 
is hundreds of years old and has been treated excellently in many famous texts. An 
undergraduate mathematics text which strove to be unlike its predecessors would 


2My conception of the meaning of deconstruction in the sense of academic humanities is 
painfully vague. I am more thinking of the term in the sense that chefs use it. 
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almost certainly do so to the detriment of its audience. 

Having said this, most undergraduate texts with “calculus” in the title offer 
very little innovation indeed. It is distressing to me how many calculus books are 
written which look like nearly exact replicas of some platonic (but far from perfect) 
calculus text written circa 1920. Apparently the issue of treating transcendental 
functions earlier rather than later was enough to warrant many new editions, if not 
wholly new texts. 

This text does contain some novelties, up to and including a proof technique, 
real induction, which to my knowledge has not appeared before in texts. Spivak’s 
text is a great innovation, and its distinctiveness — some might say eccentricity — 
has not gone unnoticed by instructors. This text is also eccentric, and in its own 
way: if you like Spivak’s text, you need not like this one. 


What is Honors Calculus? 


It was audacious of Spivak to call his text simply “calculus,” since it is light years 
away from any other text of that name currently in print. I have chosen to call this 
text: honors calculus. What is honors calculus? 


By “honors calculus” I mean a certain take on undergraduate real analysis. 


Some Features of the Text 


e Our approach is heavily theoretical but not heavily abstract. 


The jumping off point for an honors calculus course is a theoretical understanding 
of the results seen in freshman calculus, especially rigorous definitions of the various 
limiting processes and proofs of the “hard theorems” of calculus, which inevitably 
turn on the completeness of R. We are not interested in calculation for its own sake, 
and we do not give any halfway serious applications to any subject outside mathe- 
matics. This theoretical perspective is of course a huge change from the practical, 
problem-oriented approach of freshman calculus (with which we assume a prior 
familiarity). It is certainly not as appealing to as broad an audience as freshman 
calculus. At the University of Georgia, it happens that many freshman students 
(especially, those in the honors program) are signed up for Math 2400 without any 
real idea of what they are getting into. The attrition at the beginning of the course 
is therefore significant, and the students that remain are not necessarily the most 
talented or experienced but rather those that remain interested in a fully theoretical 
approach to calculus. This text assumes that interest on the part of the reader, in 
fact probably even more so than Spivak’s text. 


The idea of presenting a fully theoretical take on real analysis to a young un- 
dergraduate audience is hardly new, of course. But most contemporary texts on 
real analysis take a more sophisticated approach, making use of concepts from set 
theory and topology. In this text we rarely speak explicitly about the connected- 
ness and compactness of intervals on the real line. (More precisely, we never use 
these concepts in the main text, but only in some digressions which briefly and 
optionally present some advanced material: e.g. § 10.10.5 introduces metric spaces 
and discussed compactness therein.) Instead we make e-d arguments which amount 
to this. Let us freely admit that this concreteness makes certain arguments longer 
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and harder (or phrased more positively: one merit of abstraction is to make certain 
arguments shorter and easier). 

I am a fan of abstraction, but I have come to believe that it is much less use- 
ful — and moreover, much less educational — in basic real analysis than in most 
other branches of pure mathematics. A turning point in my feelings about this 
was a second semester undergraduate real analysis course I taught at McGill Uni- 
versity in 2005. When preparing to teach the course I was quite surprised at how 
pedestrian the syllabus seemed: an entire year of real analysis was restricted to 
the one-dimensional case, and there was no topology or set theory whatsoever.® I 
was strictly forbidden from talking about metric spaces, for instance. The topics of 
the course were: infinite series, the Riemann integral, and sequences and series of 
functions, including uniform convergence. By the end of the course I had acquired 
more respect for the deep content inherent in these basic topics and the efficacy of 
treating them using no more than e-6 arguments and the completeness axiom. This 
was also the first course I taught in which I typed up lecture notes, and portions of 
these notes appear here as parts of Chapters 11 through 14. 


Exception: We begin our treatment of the Riemann integral with an axiomatic ap- 
proach: that is, we list some reasonable properties (axioms) that an area functional 
should satisfy, and before we do the hard work of constructing such a functional 
we explore the consequences of these axioms. In particular, less than two pages 
into our discussion of integration we state and prove (given the axioms!) the Fun- 
damental Theorem of Calculus. This approach to the Riemann Integral resonates 
deeply with me, as it addresses concerns that bubbled up over several years of my 
teaching this material in the context of freshman calculus. I came up with this 
approach in late 2004 while preparing for the McGill analysis course. Just a few 
days later I noticed that Lang had done (essentially) the same thing in his text [L]. 
I was encouraged that this idea had been used by others, and I endorse it still. 


Given the choice between pounding out an ¢-d argument and developing a more 
abstract or softer technique that leads to an easier proof, in this text we usually 
opt for the former. Well, that’s not strictly true: sometimes we do both. 


e We often spend time giving multiple proofs and approaches to basic results. 


For instance, there are two distinct approaches to the Riemann integral: Riemann’s 
original approach using tagged partitions and Riemann sums, and Gaston Dar- 
boux’s later simplification using upper and lower sums and integrals. Most texts at 
this level cover only one of these in detail, but here we cover both: first Darboux, 
then Riemann. I got “permission” to do so while teaching the analysis class at 
McGill, since this was done in the official course text of R. Gordon [Go]. Later I 
realized that Gordon is in real life an integration theorist! 


More significantly, we prove the Interval Theorems using ¢-6 arguments and later 
come back to give much quicker proofs using sequences and the Bolzano-Weierstrass 
Theorem. One may well say that this is evidence that sequences should be treated 


3In fact, I didn’t find out until just after the course ended that the students did not know 
about countable and uncountable sets. Without conscious thought I had assumed otherwise. 
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at the beginning of the course rather than towards the end, and many texts do 
take this approach, most notably [R]. However I endorse Spivak’s ordering of the 
material, in which honors calculus essentially begins with a thorough grappling 
with the e-6 definition of limit. Although there are easier — and especially, softer 
— approaches to most of the key theorems, I feel that manipulation of inequalities 
that characterized hard analysis is a vitally important skill for students to learn 
and this is their best chance to learn it. 


e We view the completeness axiom as the star of the show. Thus we do not put it 
on the stage in Act I, Scene I, when people are still settling into their seats. 


CHAPTER 1 


Introduction and Foundations 


1. Introduction 


1.1. The Goal: Calculus Made Rigorous. 


The goal of this course is to cover the material of single variable calculus in a 
mathematically rigorous way. The latter phrase is important: in most calculus 
classes the emphasis is on techniques and applications; while theoretical explana- 
tions may be given by the instructor — e.g. it is usual to give some discussion of 
the meaning of a continuous function — the student tests her understanding of 
the theory mostly or entirely through her ability to apply it to solve problems. 
This course is very different: not only will theorems and proofs be presented in 
class by me, but they will also be presented by you, the student, in homework and 
on exams. This course offers a strong foundation for a student’s future study of 
mathematics, at the undergraduate level and beyond. 


As examples, here are three of the fundamental results of calculus; they are called 
— by me, at least — the three Interval Theorems, because they all concern an 
arbitrary continuous function defined on a closed, bounded interval. 


THEOREM 1.1. (Intermediate Value Theorem) Let f : [a,b] > R be a continuous 
function defined on a closed, bounded interval. Suppose that f(a) <0 and f(b) > 0. 
Then there exists c with a < c < b such that f(c) =0. 


THEOREM 1.2. (Extreme Value Theorem) Let f : [a,b] > R be a continuous 
function defined on a closed, bonuded interval. Then f is bounded and assumes its 
mazimum and minimum values: there are numbers m < M such that 
a) For all x € [a,b], m< f(x) <M. 

b) There exists at least one x € [a,b] such that f(x) =m. 
c) There exists at least one x € [a,b] such that f(x) = M. 


THEOREM 1.3. (Uniform Continuity and Integrability) Let f : [a,b] > R be a 
continuous function defined on a closed, bounded interval. Then: 
a) f is uniformly continuous. 


b) f is integrable: re exists and is finite. 
Except for the part about uniform continuity, these theorems are familiar results 
from freshman calculus. Their proofs, however, are not. Most freshman cal- 


culus texts like to give at least some proofs, so it is often the case that these three 
theorems are used to prove even more famous theorems in the course, e.g. the 


lThe definition of this is somewhat technical and will be given only later on in the course. 
Please don’t worry about it for now. 
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Mean Value Theorem and the Fundamental Theorem of Calculus. 


Why then are the three interval theorems not proved in freshman calculus? Because 
their proofs depend upon fundamental properties of the real numbers that are not 
discussed in such courses. Thus one of the necessary tasks of the present course is 
to give a more penetrating account of the real numbers than you have seen before. 


1.2. Numbers of Various Kinds. 


There are various kinds of “numbers”. Here is a list of the ones which will be 
most important to us: 


(1) ZACNCZcCQCRCC. 


Let me remind you what these various numbers are. 


ae es eee ee 


is the set of positive integers (a.k.a. “counting numbers”). On them we have 
defined the operations of addition + and multiplication -. Moreover there is an 
identity element for the multiplication, namely 1. There is no additive identity. 


N= {0} UZ* = {0,1,2,3,...,n,...} 


is the set of natural numbers. Again we have operations of addition and multi- 
plication, and now we have an additive identity as well as a multiplicative identity. 

Clearly Z* and N are very similar: they differ only as to whether 0 is included 
or not. In analysis — the subject we are beginning the study of here! — the distinc- 
tion between Z* and N is not very important, and in fact most analysts I know 
use N to denote the positive integers. I am probably showing my stripes as an 
algebraically minded mathematician by making the distinction, but so be it. 

Recall that the operation of subtraction is nothing else than the inverse opera- 
tion of addition: in other words, to say a — b = c is to say that a = b+ c. However 
the operation of subtraction is not everywhere defined on N: for instance, there is 
a natural number 5 — 3, but no natural number 3 — 5. 


| Fee Rey ed 


is the set of integers. This is formed out of the natural numbers N by formally 
allowing all subtractions: for instance, —17 is 0 — 17. Thus in Z every element n 
has an additive inverse —n. However, the same cannot be said of multiplicative 
inverses. Recall that the operation of division is nothing else than the inverse oper- 
ation of multiplication: to say a/b = cis to say that a = b-c. However the operation 
of division is not everywhere defined on Z: thre is an integer 6/3, but no integer 6/4. 


Q= {5 | a,beZ,b #0}. 


is the set of rational numbers. This is formed out of the integers Z by formally 
allowing all divisions by nonzero integers. One subtlety: the same rational num- 


ber has many different expressions as the quotient of two integers: for instance 


$ — 3 = ge for any n € Zt. So we just need to agree that ¢ = § iff ad = be. Alter- 
nately, any nonzero rational number has a unique expression ¢ in lowest terms, 
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i.e., with a and b not both divisible by any integer n > 1.2? Thus in Q we have an 
additive identity 0, every element has an additive inverse, we have a multiplicative 
identity 1, and every nonzero element has a multiplicative inverse. 


What then are the real numbers R? The geometric answer is that the real numbers 
correspond to “points on the number line”, but this does not make clear why there 
are such points other than the rational numbers. An answer that one learns in high 
school is that every real number has an infinite decimal expansion, not necessarily 
terminating or repeating, and conversely any integer followed by an infinite deci- 
mal expansion determines a real number. In fact this is perfectly correct: it gives a 
complete characterization of the real numbers, but it is not a cure-all: in order to 
pursue the implications of this definition — and even to really understand it — one 
needs tools that we will develop later in the course. 


Finally, the complex numbers C are expressions of the form a+ bi where a and 
b are real numbers and i? = —1. They are extremely important in mathematics 
generally — e.g. one needs them in order to solve polynomial equations — but in this 
course they will play at most a peripheral role. 


Back to R: let us nail down the fact that there are real numbers which are not 
rational. One way to see this is as follows: show that the decimal expansion of 
every rational number is eventually periodic, and then exhibit a decimal expansion 
which is not eventually periodic, e.g. 


x = 0.16116111611116111116... 


where the number of 1’s after each 6 increases by 1 each time. But this number x 
reeks of contrivance: it seems to have been constructed only to make trouble. The 
ancient Pythagoreans discovered a much more “natural” irrational real number. 


THEOREM 1.4. The square root of 2 is not a rational number. 


ProoF. The proof is the most famous (and surely one of the first) instances 
of a certain important kind of argument, namely a proof by contradiction. We 
assume that what we are trying to prove is false, and from that we reason until we 
reach an absurd conclusion. Therefore what we are trying to prove must be true. 

Here goes: seeking a contradiction, we suppose V2 is rational: then there are 
integers a,b with b > 0 and V2 = §. Since the defining property of V2 is that its 
square is 2, there is really nothing to do but square both sides to get 


Clearing denominators, we get 

2b? = a’. 
This shows that the integer a? is even, i.c., divisible by 2. It happens that for any 
integer a, if a? is even, then so is a: let us assume this for now; we can come back 
to it later. Thus we may write a = 2A with A € Z. Substituting, we get 


2b? =a = (2A)? = 4A?, 
2Like most of the statements we have made recently, this requires proof! Let me reiterate 


that we are not giving proofs here or even careful definitions; rather, we are reminding the reader 
of some of her mathematical past. 
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or 

b? = 2A”. 
Thus 0? is divisible by 2, so as above b = 2B for some B € Z. Substituting, we get 

4B? = (2BY =P = 227, 

or 

PS eee 
Thus we are back where we started: assuming that 2b? = a?, we found that both 
a and b were divisible by 2. This is suspect in the extreme, and we now have our 
choice of killing blow. One ending is to observe that everything we have said above 
applies to A and B: thus we must also have A = 2A;,, B = 2Bzy, and so forth. 
We can continue in this way factoring out as many powers of 2 from a and b as we 
wish. But the only integer which is arbitrarily divisible by 2 is 0, so our conclusion 
is a= b =O, whereas we assumed b > 0: contradiction. 

Alternately — and perhaps more simply — each rational number may be written 
in lowest terms, so we could have assumed this about § at the outset and, in par- 
ticular, that a and 6 are not both divisible by 2. Either way we get a contradiction, 
so V2 must not be a rational number. 


1.3. Why do we not do calculus on Q? 


To paraphrase the title question, why do we want to use R to do calculus? Is 
there something stopping us from doing calculus over, say, Q? 


The answer to the second question is no: we can define limits, continuity, deriva- 
tives and so forth for functions f : Q — Q exactly as is done for real functions. The 
most routine results carry over with no change: it is still true, for instance, that 
sums and products of continuous functions are continuous. However most of the 
big theorems — especially, the Interval Theorems — become false over Q. 


For a,b € Q, let [a,blo = {x EQ] a<a< bd}. 


EXAMPLE 1.1. Consider the function f : [0,2]q > Q given by f(x) = —1 if 
x? <2and f(x) =1 ifx? > 2. Note that we do not need to define f(x) atx = +V2, 
because by the result of the previous section these are not rational numbers. Then 
f is continuous — in fact it is differentiable and has identically zero derivative. But 
f(0) =-1<0, f(2) =1>0, and there is no c € [0,2|q such that f(c) = 0. Thus 
the Intermediate Value Theorem fails over Q. 


EXAMPLE 1.2. Consider the function: f : [0,2]g + Q given by f(x) = oy: 
Again, this function is well-defined at all points of [0,2]q because V2 is not a rational 
number. It is also a continuous function. However it is not bounded above: by 
taking rational numbers which are arbitrarily close to /2, x? —2 becomes arbitrarily 
small and thus f(x) becomes arbitarily large.’ In particular, f certainly does not 
attain a maximum value. Thus the Extreme Value Theorem fails over Q. 

Moreover, it can be shown (and will be — later) that any function on a closed, 
bounded interval which is either uniformly continuous or integrable is bounded, so 
the above function f is neither uniformly continuous nor integrable. If you have 


3We will be much more precise about this sort of thing later on. This is just an overview. 
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had second semester freshman calculus, you should think about why the analogous 
function f : [0,2] \ {V2} > R is not improperly Riemann integrable: it builds up 
infinite area as we approach V2. 


The point of these examples is in order to succeed in getting calculus off the ground, 
we need to make use of some fundamental property of the real numbers not pos- 
sessed by (for intance) the rational numbers. This property, which can be expressed 
in various forms, is called completeness, and will play a major role in this course. 


2. Some Properties of Numbers 


2.1. Axioms for a Field. 


In order to do mathematics in a rigorous way, one needs to identify a starting 
point. Virtually all mathematical theorems are of the form A = > B. That is, 
assuming A, B must follow. For instance, in Euclidean geometry one lays down a 
set of axioms and reasons only from them. The axioms needed for calculus are a 
lot to swallow in one dose, so we will introduce them gradually. What we give here 
is essentially a codification of high school algebra, including inequalities. 


Specifically, we will give axioms that we want a number system to satisfy. At 
this point we will take it for granted that in our number system we have operations 
of addition, multiplication and an inequality relation <, and that there are distin- 
guished numbers called 0 and 1. We require the following properties: 


)OA1. 
) (Commutativity of +): For all numbers z,y, 7+y=y+. 
) (Associativity of +): For all numbers x,y,z, (@+y)+2=2+4+ (y+ 2). 
) (identity for +): For all numbers z, 7+ 0= cz. 
4) (Inverses for +): For all numbers z, there exists y with x+y = 0. 
) 
) 
) 
) 
) 


Commutativity of -): For all numbers x,y, 7-y = y- a. 

Associativity of -): For all numbers 2, y, z (@-y): z= a> (y-z). 

Identity for -): For all numbers x, «-1= 2. 

Inverses for -) For all numbers «x ¥ 0, there exists a number y with ry = 1. 
Distributivity of - over +): For all numbers 2, y, z, 2-(yt+z) = (x-y)+(a-z). 


Although it is not important for us now, the above axioms (PO) through (P9) 
are called the field axioms, and a structure which satisfies them is called a field. 


EXAMPLE 1.3. Both Q and R satisfy all of the above field axioms. (We take 
this as “known” information.) 


EXAMPLE 1.4. The complex numbers C satisfy all of the above field axioms. The 
only one which is not straightforward is the existence of multiplicative inverses. For 
this: if z= a+ ty is a nonzero complex number — i.e., the real numbers x and y 

zL—-t 


are not both zero — then ifw = raat) zw=1. 


EXAMPLE 1.5. Let Fo = {0,1} be a set consisting of two elements, 0 and 1. 
We define0+0=0,04+1=14+0=1,14+1=0,0-0=0-1=1-0=0,1-1=1. 
Then F2 satisfies all of the above field axioms. It is sometimes called the binary 
field. 
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PROPOSITION 1.5. In every system satisfying the field axioms, for every number 
x we have x-0=0. 


Proor. We have «-0=2- (04+ 0) = (a -0)+(a-0). Subtracting (# - 0) from 
both sides gives 0 = x - 0. 


PROPOSITION 1.6. In every system satisfying the field axioms: 
a) The only additive identity is 0. 
b) Every number x has a unique additive inverse. If —1 denotes the additive inverse 
of 1, then the additive inverse of x is (—1)-a. 
c) The only multiplicative identity is 1. 
b) Every nonzero number has a unique multiplicative inverse. 


PRooF. a) Note that 0 is an additive identity by (P3). Suppose that z is 
another additive identity, and consider 0+z. Since 0 is an additive identity, 0+z = z. 
Since z is an additive identity, 0+ z= 0. Thus z = 0. 

b) Suppose y and z are both additive inverses to z: e+y=2+2=0. Adding y 
to both sides gives 


y=O+y=(e@+y)ty=(ta)ty=yt (ety) 
=yt(et+z)=(yt+a) +2 
so y = z. Moreover, for any number 2, 
(-1)-e¢+e¢=((-1)-2)+(1-2)=(-14+1)-c=0-2=0. 
c), d) The proofs of these are the same as the proofs of parts a) and b) but with 
all instances of + replaced by - and all instances of 0 replaced by 1. 
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PROPOSITION 1.7. In every system satisfying the field axioms, (—1)? = 1. 


PrRooF. By Proposition 1.6, —1-—1 is the additive inverse of —1, namely 1. 


PROPOSITION 1.8. In every system satisfying the field axioms, if x # 0 and 
y £0 then ry £0. 


Proor. Seeking a contradiction, suppose that ry = 0. Since x ¥ 0 it has a 
multiplicative inverse 2~! and then by Proposition 1.5, 


O=a' -ay=(a7'-a)\y=l-y=y, 


contradicting the assumption that y 4 0. 


Note that a logically equivalent formulation of Proposition 1.8 is: in any system 
satisfying the field axioms, if zy = 0 then « = 0 or y = 0. 


2.2. Axioms for an ordered field. 


The remaining properties of numbers concern the inequality relation <. Instead 
of describing the relation < directly, it turns out to be simpler to talk about the 
properties of positive numbers. If we are given the inequality relation <, then we 
say that x is positive if x > 0, thus knowing < we know which numbers are posi- 
tive. Conversely, suppose that we have identified a subset P of numbers as positive. 
Then we can define x < y if y— x2 € P. Now we want our set of positive numbers 
to satisfy the following properties. 


(P10) (Trichotomy) For all numbers z, exactly one of the following holds: x = 0, « 
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is positive, —x is positive. 
(P11) (Closure under +) For all positive numbers x,y, x+y € P. 
(P12) (Closure under -) For all positive numbers x,y, x-y € P. 


A number system satisfying (P1) through (P12) is called an ordered field. 


PROPOSITION 1.9. In every ordered field: 
a) 1>0 and-1<0. 
b) For every nonzero x, x? > 0. 
c) It follows that for all x, x? > 0. 


PRooF. a) By (PO), 1 4 0. Thus by trichotomy, either 1 is positive and —1 
is negative, or —1 is positive and 1 is negative. But by (P12) the product of two 
positive numbers is positive, so if —1 is positive and 1 is negative then 1 = (—1)? 
is positive, a contradiction. So it must be that 1 is positive and —1 is negative. 

b) Since x is nonzero, either x > 0 or —x > 0. If x > 0, then 2? = z- z is the 
product of two positive numbers, hence positive. If x < 0, then —xz > 0 and then 
x? = (—1)?a? = (—a)-(—2) is the product of two positive numbers, hence positive. 
c) Since 0? = 0, part c) follows immediately from part b). 


EXAMPLE 1.6. The binary numbers Fp satisfy the field axioms (PO) through 
(P9), but are they an ordered field? Well, not on the face of it because we have not 
been given an inequality relation < satisfying (P10) through (P12). In fact we will 
now show that there is no such relation. Indeed, in any ordered field, since 1 > 0, 
alsol+1>0, but in Fo 1+1=0. In fancy language, F2 is a field which cannot 
be endowed with the structure of an ordered field. 


EXAMPLE 1.7. The complex numbers C satisfy the field axioms (PO) through 
(P9), but are they an ordered field? As above, we have not been given an inequality 
relation. Also as above we can show that there is no such relation. For in the 
complex numbers we have an element i with i? = —1. But the ordered field axioms 
imply both that —1 is negative and that any square is non-negative, contradiction. 


PROPOSITION 1.10. For any x,y,z in an ordered field: 
alx<0 = 0<-—a. (We say “x is negative”.) 
b) The trichotomy property may be restated as: for any number x, exactly one of 
the following holds: x is positive, x is zero, x is negative. 
c) If x ts positive, 4 is positive. If x is negative, 4 is negative. 
d) If x is positive and y is negative, then xy is negative. 


e) If x and y are both negative, then xy is positive. 


PRooF. a) By definition, x < 0 means 0 — x = —z is positive. Also 0 < —ax 
means —x — 0 = —2 is positive. So there is nothing to show here. 
b) No further argument for this is needed; we just state it for future reference. 
c) Suppose x is positive. Certainly - is not zero, so we need to rule out the 
possibility that it’s negative. But if it were, then by part a) = would be positive 
and thus by (P12) a- =1 = —1 would be positive, contradicting Proposition 1.9a). 
If x is negative then —z is positive so by what we just showed + = =1 is positive, 
and thus + = —(=") is negative. 
d) Suppose x is positive and y is negative. In particular x and y are not zero, 
so cy # 0. To show that ry is negative, by part b) it is enough to rule out the 
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possibility that xy is positive. Suppose it is. Then, by part c), since x is positive, 
4 is positive, and thus y = zy- 4 would be positive: contradiction. 

e) Suppose x and y are both negative. Then xy 4 0, and we need to rule out the 
possibility that ry is negative. Suppose it is. Then —zy is positive, 4 is negative, 
so by part d) —y = —ay- 4 is negative and thus y is positive: contradiction. 


PROPOSITION 1.11. For alla,b,c,d in an ordered field: 
a) Ifa<bandc<d,a+c<b+d. 
b) Ifa<b andc> 0, then ac < be. 
c) Ifa<b andc <0, then ac > be. 
d) If0<a<b, then0<} <2. 
e) Ifa>0 andb>0, thena<b => a? <D?. 


PROOF. a) Since a < b and c < d, b —a is positive and d —c is positive, and 
then by (P11) (6— a) + (d—c) = (b+ d) — (a+¢) is positive, sob+d>a+te. 
b) Since a < b, b—a is positive. Since c is positive, by (P12) be — ac = (b—c)a is 
positive, so bc > ac. 
c) Left to the reader. 
d) We have 7 i =(b-a)- 4. The hypotheses imply that 6 — a and “ are both 
positive, so by (P12) so is their product. 
e) Note that b? — a? = (b+ a)(b—a). Since a and B are both positive, b + a is 
positive, and therefore b — a is positive iff b? — a? is positive. 


2.3. Some further properties of Q and R. 


As we have mentioned before, the ordered field axioms (PO) through (P12) are 
just a list of some of the useful properties of Q and R. They are not a “complete 
set of axioms” for either Q or R — in other words, there are other properties these 
fields have that cannot be logically deduced from these axioms alone. In fact this 
is already clear because Q and R each satisfy all the ordered field axioms but are 
essentially different structures: in Q the element 2 = 1+ 1 is not the square of 
another element, but in R it is. Here we want to give some further “familiar” prop- 
erties that do not hold for all ordered fields but both of which hold for R and one 
of which holds for Q. (We are still far away from the fundamental completeness 
axiom for R which is necessary to prove the Interval Theorems.) 


The first axiom is called the Archimedean property: it says that for any positive 
number wz, there is a positive integer n such that « <n. This clearly holds for R 
according to our description of real numbers as integers followed by infinite decimal 
expansions: a positive real number z is of the form 


LZ = N9-.41AQ...An..-- 


with no € Zt and a; € {0,1,2,3,4,5,6,7,8,9}, and thus z is less than or equal to 
the integer ng + 1.4 


Since every positive real number is less than or equal to some integer, and ev- 
ery positive rational number is, in particular, a positive real number, then also 
every positive rational number is less than or equal to some integer. That is, Q also 


4When I first typed this I wrote that x is less than no + 1. But actually this need not be 
true! Can you think of an example? Beware: decimal expansions can be tricky. 
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satisfies the Archimedean property. (Or, directly: any positive rational number 
a 


may be written in the form ¢ with a,b € Z*, and then tS a.) 
This Archimedean property is so natural and familiar (not to mention useful...) 
that the curious student may be well wonder: are there in fact systems of numbers 
satisfying the ordered field axioms but not the Archimedean property?!? The an- 
swer is yes, there are plenty of them, and it is in fact possible to construct a theory 
of calculus based upon them (in fact, such a theory is in many ways more faithful to 
the calculus of Newton and Leibniz than the theory which we are presenting here, 
which is a 19th century innovation). But we will not see such things in this course! 


The next property does provide a basic difference between Q and R. 


THEOREM 1.12. Let x be a real number andn € Z*. 
a) If n is odd, there is a unique real number y such that y” =a. We writey = Wa. 
b) If n is even and x is positive, there is a unique positive real number y such that 
y" =a. We writey= Ya. 
c) If n is even and x is negative, then there is no real number y with y” = x. 


The first two parts of Theorem 1.12 rely on the Intermediate Value Theorem so are 
not accessible to us at this time. (Thus we must guard aginst using the existence of 
nth roots of real numbers in any of the theorems that lead up to the Intermediate 
Value Theorem. In fact we will not use such things in the proof of any theorem, 
but only as examples.) As a supplement tto part b), note that if n is even and 
y is a positive real number such that y” = x, then there is exactly one other real 
number with nth power equal to 7: —y. (You might try to prove this as an exercise.) 


We can prove part c) now, since it holds in any ordered field. Indeed, if n is 
even then n = 2k for an integer k, so if for some negative x we have y” = x then 
x = y?* = (y*)?, contradicting Proposition 1.9c). 


Here is a special case of Theorem 1.12 important enough to be recorded separately. 


COROLLARY 1.13. A real number x is non-negative if and only if it is a square, 
i.e., if and only if there exists a real number y with y? = x. 


Note that Corollary 1.13 does not hold in the number system Q, since 2 = 1+ 1 is 
positive but is not the square of any rational number. 


Corollary 1.13 leads to a basic strategy for proving inequalities in the real numbers: 
forz,y ER, 2 <y — > (y—2) = 2? for some real number z. In the next section 
we will see some instances of this strategy in action. 


2.4. Some Inequalities. 


For an element x of an ordered field, we define the absolute value of x to be 
x if x > 0 and —z if x < 0; it is denoted by |x|. Thus |z| > 0 always and « = +|2|. 


PROPOSITION 1.14. For any number x in an ordered field, x < |x|. 


Proor. If z >0 then ¢ = |z|. Ife <0 then <0< —r=|a|, soz < |a|. 


THEOREM 1.15. (Triangle Inequality) For all numbers x,y, |x + y| < |x| + |y|. 
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PROOF. Since |x| is defined to be x if 2 > 0 and —z if x < 0, it is natural to 
break the proof into cases. 
Case 1: x,y > 0. Then |x+y|=x2+y= |2|+ yl. 
Case 2: x,y < 0. Thenz+y<0,s0 |jv+y|=—-(e+y) =-—2-—y= |x| + yl. 
Case 3: x > 0, y < 0. Now unfortunately we do not know whether |x + y| is 
non-negative or negative, so we must consider consider further cases. 
Case 8a: x+y >0. Then |x+y|=r+y < |a| + lyl. 
Case 3b: e+ y <0. Then |x+y| =—-2-y<|-—a2]+]—y|=|2|+lyl. 
Case 4: « < 0, y > 0. The argument is exactly the same as that in Case 3. In fact, 
we can guarantee it is the same: since the desired inequality is symmetric in x 
and y — meaning, if we interchange x and y we do not change what we are trying 
to show — we may reduce to Case 3 by interchanging x and y.° 


The preceding argument is definitely the sort that one should be prepared to make 
when dealing with expressions involving absolute values. However, it is certainly 
not very much fun. Spivak gives an alternate proof of the Triangle Inequality 
which is more interesting and thematic. First, since both quantities |a + y| and 
|x| + |y| are non-negative, the inequality will hold iff it holds after squaring both 
sides (Proposition 1.1le). So it is enough to show 


(|x + yl)? < (le + [yl)?. 


now (a+y|)? = (x+y)? = 2° +2axy+y?, whereas (|a|+|y|)? = |a|?+2|e|ly|+|yl? = 
x? + |2ry| + y?, so subtracting the left hand side from the right, it is equivalent to 
show that 


O< (2 + |2xy| + y*) — (a? + 2ay +-y7). 
But 


(a? + |2xy| + y? — (a? + 2xy + y*) = |2ary| — 2ay > 0 
by Proposition 1.14. So this gives a second proof of the Triangle Inequality. 
A similar argument can be used to establish the following variant. 


PROPOSITION 1.16. (Reverse Triangle Inequality) 
For all numbers x,y, ||x| — |y|| < |a — yl. 


Proor. Again, since both quantities are non-negative, it is sufficient to prove 
the inequality after squaring both sides: 


(Ilee] — fall)? = (el — lyl)? = |e? — 2lellyl + ly? = 2? — [2ay| + 9? 


<a? — Ay +y? = (x —y)? = (\2-yl)?. 


EXERCISE 1.1. Let x,y be any numbers. 
a) Show that |x| — |y| < |a — y| by writing « = (a — y) + y and applying the usual 
triangle inequality. 
b) Deduce from part a) that ||x| — |y|| < la — yl. 
THEOREM 1.17. (Cauchy-Bunyakovsky-Schwarz Inequality, n = 2) 
a) For all numbers x1, £2, yi, Ya; 


(2) (xiy1 + ayo)” < (xt + ©3)(yp + 93). 


5Such symmetry arguments can often by used to reduce the number of cases considered. 
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b) Moreover equality holds in (2) iff y: = y2 = 0 or there exists a number X such 
that x, = Ay, and x2 = Ayo. 
PROoF. By pure brute force, one can prove the two squares identity: 
| 


(xt + ©3)(yi + y3) = (eiye — tay)? + (1491 + 2y2)?. 


Now we need only rewrite it in the form 


(ai + 23)(yt + 93) — (ery + waye)” = (e1y2 — ray)” > 0, 
establishing part a). Moreover, equality holds iff r1y2 = reyi. If in this equality 
yi and y2 are both nonzero, we may divide by them to get a = aa =x. Ify, =0 
and y2 # 0 then we must have x; = 0 and then we may take \ = me Similarly, 
if y: A 0 and yg = 0, then we must have zz = 0 and then we may take \ = a 
Finally, if y1 = y2 = 0 then the equality x1 y2 = xray also holds. 


THEOREM 1.18. (Cauchy-Bunyakovsky-Schwarz Inequality) For any n € Z* 
and numbers £1,..-,2n;Y1;---,Yn we have 


(my. +... + 2nYn)? < (a2 +... +22 )(y? +... +2). 
Proor. Expanding out the right and left hand sides, we get 


RHS = z xey? + ss U7Y;. 
i=l ij 


LHS = 3 xey? +2 ye LiYil Yj, 
i=l 


t<j 


RHS — LHS = sy ay; — >. LiYjL Yi = (ey; — xjyi)? > 0. 


iA i<j i<j 


so 


THEOREM 1.19. (Arithmetic-Geometric Mean Inequality, n = 2) 
For all numbers 0 <a < b, we have 


b 2 
a<ab< (+) <b. 


PROOF. First inequality: Since a >0 and0<a<b,a-a<a-b. 
Second inequality: Expanding out the square and clearing denominators, it is equiv- 
alent to 4ab < a? + 2ab + ab?, or to a? — 2b+0? > 0. But a? — 2ab +b? = (a — b)?, 


so since a # b, (a—b)? > 0. 
Third inequality: Since ath and 0 are both positive, it is equivalent to are < band 


thus to a+b < 2b. But indeed since a < b,a+b<b+b= 20. 


Later we will use the theory of convexity to prove a signficant generalization of 
Theorem 1.19, the Weighted Arithmetic-Geometric Mean Inequality. 


CHAPTER 2 


Mathematical Induction 


1. Introduction 


Principle of Mathematical Induction for sets 

Let S be a subset of the positive integers. Suppose that: 
(i) 1 € S$, and 

(ii) Vne Zt neS = nt+1e58. 

Then S$ =Z?*. 


The intuitive justification is as follows: by (i), we know that 1 € S. Now ap- 
ply (ii) with n = 1: since 1 € S, we deduce 1+ 1 = 2 € S. Now apply (ii) with 
n = 2: since 2 € S, we deduce 2+ 1=3 € S. Now apply (ii) with n = 3: since 
3 € S, we deduce 3+ 1=4€S. And so forth. 


This is not a proof. (No good proof uses “and so forth” to gloss over a key point!) 
But the idea is as follows: we can keep iterating the above argument as many times 
as we want, deducing at each stage that since S contains the natural number which 
is one greater than the last natural number we showed that it contained. Now it 
is a fundamental part of the structure of the positive integers that every positive 
integer can be reached in this way, i.e., starting from 1 and adding 1 sufficiently 
many times. In other words, any rigorous definition of the natural numbers (for 
instance in terms of sets, as alluded to earlier in the course) needs to incorporate, 
either implicitly or (more often) explicitly, the principle of mathematical induction. 
Alternately, the principle of mathematical induction is a key ingredient in any ax- 
iomatic characterization of the natural numbers. 


It is not a key point, but it is somewhat interesting, so let us be a bit more specific. 
In Euclidean geometry one studies points, lines, planes and so forth, but one does 
not start by saying what sort of object the Euclidean plane “really is”. (At least 
this is how Euclidean geometry has been approached for more than a hundred years. 
Euclid himself gave such “definitions” as: “A point is that which has position but 
not dimensions.” “A line is breadth without depth.” In the 19th century it was 
recognized that these are descriptions rather than definitions, in the same way that 
many dictionary definitions are actually descriptions: “cat: A small carnivorous 
mammal domesticated since early times as a catcher of rats and mice and as a pet 
and existing in several distinctive breeds and varieties.” This helps you if you are 
already familiar with the animal but not the word, but if you have never seen a cat 
before this definition would not allow you to determine with certainty whether any 
particular animal you encountered was a cat, and still less would it allow you to 
reason abstractly about the cat concept or “prove theorems about cats.”) Rather 
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“point”, “line”, “plane” and so forth are taken as undefined terms. They are 
related by certain axioms: abstract properties they must satisfy. 


In 1889, the Italian mathematician and proto-logician Gisueppe Peano came up 
with a similar (and, in fact, much simpler) system of axioms for the natural num- 
bers. In slightly modernized form, this goes as follows: 


The undefined terms are zero, number and successor. 


There are five axioms that they must satisfy, the Peano axioms. The first four 
are: 

(P1) Zero is a number. 

(P2) Every number has a successor, which is also a number. 

(P3) No two distinct numbers have the same successor. 

(P4) Zero is not the successor of any number. 


Using set-theoretic language we can clarify what is going on here as follows: the 
structures we are considering are triples (X,0,S), where X is a set, 0 is an element 
of X, and S: X > X is a function, subject to the above axioms. 


From this we can deduce quite a bit. First, we have a number (i.e., an element 
of X) called S(0). Is 0 = S(0)? No, that is prohibited by (P4). We also have a 
number 5(S(0)), which is not equal to 0 by (P4) and it is also not equal to S(0), 
because then $(0) = S(S(0)) would be the successor of the distinct numbers 0 
and $(0), contradicting (P3). Continuing in this way, we can produce an infinite 
sequence of distinct elements of X: 


(3) 0, 5(0), $($(0)), $($(S(0)),.... 


In particular X itself is infinite. The crux of the matter is this: is there any element 
of X which is not a member of the sequence (3), i.e., is not obtained by starting at 
0 and applying the successor function finitely many times? 


The axioms so far do not allow us to answer this question. For instance, suppose 
that the “numbers” consisted of the set [0, co) of all non-negative real numbers, we 
define 0 to be the real number of that name, and we define the successor of x to be 
x +1. This system satisfies (P1) through (P4) but has much more in it than just 
the natural numbers we want, so we must be missing an axiom! Indeed, the last 
axiom is: 


(P5) If Y is a subset of the set X of numbers such that 0 € Y and such that 
x €Y implies S(x) € Y, then Y = X. 


Notice that the example we cooked up above fails (P5), since in [0,00) the subset 
of natural numbers contains zero and contains the successor of each of its elements 
but is a proper subset of [0, 00). 


Thus it was Peano’s contribution to realize that mathematical induction is an ax- 
iom for the natural numbers in much the same way that the parallel postulate is 
an axiom for Euclidean geometry. 
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On the other hand, it is telling that this work of Peano is little more than one 
hundred years old, which in the scope of mathematical history is quite recent. 
Traces of what we now recognize as induction can be found from the mathematics 
of antiquity (including Euclid’s Elements!) on forward. According to the (highly 
recommended!) Wikipedia article on mathematical induction, the first mathemati- 
cian to formulate it explicitly was Blaise Pascal, in 1665. During the next hundred 
years various equivalent versions were used by different mathematicians — notably 
the methods of infinite descent and minimal counterexample, which we shall dis- 
cuss later — and the technique seems to have become commonplace by the end of 
the 18th century. Not having an formal understanding of the relationship between 
mathematical induction and the structure of the natural numbers was not much 
of a hindrance to mathematicians of the time, so still less should it stop us from 
learning to use induction as a proof technique. 


Principle of mathematical induction for predicates 

Let P(x) be a sentence whose domain is the positive integers. Suppose that: 
(i) P(1) is true, and 

(ii) For alln € Zt, P(n) is true => P(n+ 1) is true. 

Then P(n) is true for all positive integers n. 


Variant 1: Suppose instead that P(2) is a sentence whose domain is the natu- 
ral numbers, i.e., with zero included, and in the above principle we replace (i) by 
the assumption that P(0) is true and keep the assumption (ii). Then of course the 
conclusion is that P(n) is true for all natural numbers n. This is more in accordance 
with the discussion of the Peano axioms above.! 


EXERCISE 2.1. Suppose that No is a fixed integer. Let P(x) be a sentence whose 
domain contains the set of all integers n > No. Suppose that: 
(i) P(No) is true, and 
(ii) For alln > No, P(n) is true => P(n+1) is true. 
Show that P(n) is true for all integers n > No. (Hint: define a new predicate Q(n) 
with domain Z* by making a “change of variables” in P.) 


2. The First Induction Proofs 
2.1. The Pedagogically First Induction Proof. 


There are many things that one can prove by induction, but the first thing that 
everyone proves by induction is invariably the following result. 


PROPOSITION 2.1. For alln € Z*+,14+...+n= Mt), 


PROOF. We go by induction on n. 
Base case (n = 1): Indeed 1 = 1D) 


Induction step: Let n € Z* and suppose that 1+...+n= ninth) Then 
1 
1+...¢n4tn4+1=(14 tnj¢ng1 SMO) pays 


ln fact Peano’s original axiomatization did not include zero. What we presented above is a 
standard modern modification which is slightly cleaner to work with. 
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non Qn+2_ n?+2n 3 (n+1)(n4+2) (n+1)((n+1)+1) 


2 3 ~ 2 = 2 
Here the letters “IH” signify that the induction hypothesis was used. 


Induction is such a powerful tool that once one learns how to use it one can prove 
many nontrivial facts with essentially no thought or ideas required, as is the case in 
the above proof. However thought and ideas are good things when you have them! 
In many cases an inductive proof of a result is a sort of “first assault” which raises 
the challenge of a more insightful, noninductive proof. This is certainly the case 
for Proposition 2.1 above, which can be proved in many ways. 


Here is one non-inductive proof: replacing n by n — 1, it is equivalent to show: 

(n —1)n 

We recognize the quantity on the right-hand side as the binomial coefficient (Oy: 
it counts the number of 2-element subsets of an n element set. This raises the 
prospect of a combinatorial proof, i.e., to show that the number of 2-element 
subsets of an n element set is also equal to 1+2+...+n-—1. This comes out 


(4) Vne€Zn>2:14+...¢n-1= 


immediately if we list the 2-element subsets of {1,2,...,n} in a systematic way: 
we may write each such subset as {i,j} with 1<i<n-—landi<j<n. Then: 


The subsets with least element 1 are {1,2}, {1,3},..., {1,n}, a total of n — 1. 
The subsets with least element 2 are {2,3}, {2,4},...,{2,n}, a total of n — 2. 


The subset with least element n — 1 is {n —1,n}, a total of 1. 


Thus the number of 2-element subsets of {1,...,n} is on the one hand (3) and 
on the other hand (n — 1) + (n—2)+...41=1+2+4+...+n-1. This gives a 
combinatorial proof of Proposition 2.1. 

2.2. The (Historically) First(?) Induction Proof. 


THEOREM 2.2. (Euclid) There are infinitely many prime numbers. 


Proor. For n € Z™, let P(n) be the assertion that there are at least n prime 
numbers. Then there are infinitely many primes if and only if P(n) holds for all 
positive integers n. We will prove the latter by induction on n. 

Base Case (n = 1): We need to show that there is at least one prime number. For 
instance, 2 is a prime number. 

Induction Step: Let n € Z*, and assume that P(n) holds, i.e., that there are at 
least n prime numbers p; < ...< pn. We need to show that P(n +1) holds, ice., 
there is at least one prime number different from the numbers we have already 
found. To establish this, consider the quantity 


Since pi ---Pn > pi > 2, Ny, > 3. In particular it is divisible by at least one prime 
number, say q.” But I claim that N,, is not divisible by p; for any 1 <i <n. Indeed, 
if Ny = ap; for some a € Z, then let b = ree € Z. Then kp; = pi--- Pn +1 = 


2Later in these notes we will prove the stronger fact that any integer greater than one may 
be expressed as a product of primes. For now we assume this (familiar) fact. 
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bp; +1, so (k — b)p; = 1 and thus p; = +1, a contradiction. So if we take q to be, 
for instance, the smallest prime divisor of N,,, then there are at least n + 1 prime 
numbers: p1,.--,DPn;4q- 


The proof that there are infinitely many prime numbers first appeared in Euclid’s 
Elements (Book IX, Proposition 20). Euclid did not explicitly use induction, but in 
retrospect his proof is clearly an inductive argument: what he does is to explain, as 
above, how given any finite list p,...,p, of distinct primes, one can produce a new 
prime which is not on the list. (Euclid does not verify the base case; he must have 
regarded it as obvious that there is at least one prime number.) What is strange is 
that in our day Euclid’s proof is generally not seen as a proof by induction. Rather, 
it is often construed as a proof by contradiction — which it isn’t! Rather, Euclid’s 
argument is perfectly constructive. Starting with any given prime number — say 
p, = 2 — and following his procedure, one generates an infinite sequence of primes. 
For instance, N, = 2+ 1 = 3 is prime, so we take pp = 3. Then No = 2-:34+1=7 
is again prime, so we take p3 = 7. Then N3 = 2-3-7+1 = 43 is also prime, so we 
take py = 43. But this time something more interesting happens: 


Ng =2-3-7-483+1= 18-1389 


is not prime. For definiteness let us take ps to be the smallest prime factor of N4, 
So ps = 13. In this way we generate an infinite sequence of prime numbers — so the 
proof is unassailably constructive. 

By the way, this sequence of prime numbers is itself rather interesting. It 
is often called the Euclid-Mullin sequence, after Albert A. Mullin who asked 
questions about it in 1963 [Mu63]. The next few terms are 


53, 5, 6221671, 38709183810571, 139, 2801, 11, 17, 5471, 52662739, 23003, 
30693651606209, 37, 1741, 1313797957, 887, 71, 7127, 109, 23,.... 


Thus one can see that it is rather far from just giving us all of the prime numbers 
in increasing order! In fact, since to find p,+1 we need to factor N, = p1--- Pn +1, 
a quantity which rapidly increases with n, it is in fact quite difficult to compute the 
terms of this sequence, and as of 2010 only the first 47 terms are known. Perhaps 
Mullin’s most interesting question about this sequence is: does every prime num- 
ber appear in it eventually? This is an absolutely open question. At the moment 
the smallest prime which is not known to appear in the Euclid-Mullin sequence is 31. 


Remark: Some scholars have suggested that what is essentially an argument by 
mathematical induction appears in the later middle Platonic dialogue Parmenides, 
lines 149a7-c3. But this argument is of mostly historical and philosophical interest. 
The statement in question is, very roughly, that if n objects are placed adjacent 
to another in a linear fashion, the number of points of contact between them is 
n—1. (Maybe. To quote wikipedia: “It is widely considered to be one of the 
more, if not the most, challenging and enigmatic of Plato’s dialogues.”) There is 
not much mathematics here! Nevertheless, for a thorough discussion of induction 
in the Parmenides the reader may consult [Ac0O] and the references cited therein. 


3. Closed Form Identities 


The inductive proof of Proposition 2.1 is a prototype for a certain kind of 
induction proof (the easiest kind!) in which P(n) is some algebraic identity: say 
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LHS(n) = RHS(n). In this case to make the induction proof work you need only 
(i) establish the base case and (ii) verify the equality of successive differences 


LHS(n +1) — LHS(n) = RHS(n +1) — RHS(n). 


We give two more familiar examples of this. 


PROPOSITION 2.3. For alln € Z*,14+3+...+(2n-1)=n?. 


Proor. Let P(n) be the statement “1+3+...+(2n—1) =n?”. We will show 
P(n) holds for all n € Z* by induction on n. Base case (n = 1): indeed 1 = 1?. 
Induction step: Let n be an arbitrary positive integer and assume P(n): 


(5) 14+3+...+(2n-1)=n?. 
Adding 2(n + 1) — 1 = 2n + 1 to both sides, we get 
(1+3+...+(2n-1)+2(n4+1)-Ll=n?+2(n41)-Ll=n?+2n+1=(n+1)’. 


This is precisely P(n + 1), so the induction step is complete. 


PROPOSITION 2.4. For alln€ Zt, 2 +2? +... 4n? = Bervent) | 


PRooF. By induction on n. 
Base case: n = I. 
Induction step: Let n € Zt and suppose that 12 +...+n? = aint Gney) Then 


tH n(n + 1)(2n + 1) 


1t...¢r4+(n+12 2 - t(n+1P = 
Qn? + 8n?+n+64+6n?+12n+1  2n?+9n?+138n+7 
6 7 6 


On the other hand, expanding out tebe) rUA(n Ray) , we also get BrP +On7 t13n47 


Often a non-inductive proof, when available, offers more insight. Again returning 
to our archetypical example: 1+...+, it is time to tell the story of little Gauss. 
As a child of no more than 10 or so, Gauss and his classmates were asked to add up 
the numbers from 1 to 100. Most of the students did this by a laborious calculation 
and got incorrect answers in the end. Gauss reasoned essentially as follows: put 


Sp =1+...¢(n-l1)tn. 


Of course the sum is unchanged if we we write the terms in descending order: 


S,=n+t(n—-1)+...4+241. 
Adding the two equations gives 
28, =(n+1)+(nt+1)+...4+(n+1)=n(n+1), 
so 
S, = os 1) 


This is no doubt preferable to induction, so long as one is clever enough to see it. 


Mathematical induction can be viewed as a particular incarnation of a much more 
general proof technique: try to solve your problem by reducing it to a previously 
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solved problem. A more straightforward application of this philosophy allows us to 
deduce Proposition 2.3 from Proposition 2.1: 
n 


L484. Fmt) = @i-1) = 2-1 = 2 (A ~) n=ntn—n =n?. 


i=1 


4. Inequalities 


PROPOSITION 2.5. For alln€ N, 2” >n. 


PROOF ANALYSIS For n € N, let P(n) be the statement “2” > n”. We want to 
show that P(n) holds for all natural numbers n by induction. 
Base case: n = 0: 2°=1>0. 


Induction step: let n be any natural number and assume P(n): 2" > n. Then 
Dae Oe OF Doge 

We would now like to say that 2n > n+ 1. But in fact this is true if and only 

ifn > 1. Well, don’t panic. We just need to restructure the argument a bit: we 


verify the statement separately for n = 0 and then use n = 1 as the base case of 
our induction argument. Here is a formal writeup: 


PROoF. Since 2° = 1 > 0 and 2! = 2 > 1, it suffices to verify the statement 
for all natural numbers n > 2. We go by induction on n. 
Base case: n = 2: 2? =4> 2. 
Induction step: Assume that for some natural number n > 2, 2” > n. Then 


tt 2.9 > %Mm>nHt1. 


PROPOSITION 2.6. There exists No € Z*+ such that for alln > No, 2” >n°. 


PROOF ANALYSIS A little experimentation shows that there are several small values 
of n such that 2” < n?: for instance 29 = 512 < 9? = 729. On the other hand, it 
seems to be the case that we can take No = 10: let’s try. 

Base case: n = 10: 2!° = 1024 > 1000 = 10°. 


Induction step: Suppose that for some n > 10 we have 2” > n°. Then 
StS Sn 
Our task is then to show that 2n3 > (n+ 1)? for all n > 10. (By considering limits 
as n —> ov, it is certainly the case that the left hand side exceeds the right hand 
side for all sufficiently large n. It’s not guaranteed to work for n > 10; if not, we 
will replace 10 with some larger number.) Now, 
2n3 — (n +1)? = 2n3 — n3 — 3n? — 38n —1 =n? —3n? —-3n-120 


<> n? —3n?—3n>1. 


Since everything in sight is a whole number, this is in turn equivalent to 
n? — 3n? —3n > 0. 
Now n? — 3n? — 3n = n(n? — 3n — 3), so this is equivalent to n? — 3n — 3 > 0. 


The roots of the polynomial x? — 3x — 3 are x = S£V21 so n? — 38n —3 > 0 if 
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n>4= S+v25 > S+v2I In particular, the desired inequality holds if n > 10, so 
by induction we have shown that 2” > n° for all n > 10. 


We leave it to to the student to convert the above analysis into a formal proof. 


Remark: More precisely, 2” > n? for all natural numbers n except n = 2,3, 4,6, 7, 8, 9. 
It is interesting that the desired inequality is true for a little while (i.e., at n = 0,1) 
then becomes false for a little while longer, and then becomes true for all n > 10. 
Note that it follows from our analysis that if for any N > 4 we have 2% > N3, then 
this equality remains true for all larger natural numbers n. Thus from the fact that 
2° < 93, we can in fact deduce that 2” < n? for all 4<n<8. 


PROPOSITION 2.7. For allné Z+,1+4+...¢4<2-4. 
PROOF ANALYSIS By induction on n. 
Base case (n = 1): 1 < 2-4. 
Induction step: Assume that for some n € Z*, 14 + +... s <2 1. Then 
1 1 1 a 1 
14 Pele at t ee”, t : 
4 n? (n+1)? — n  (n+1)? 
We want the left hand side to be less than 2 — =H? so it will suffice to show 
1 1 1 
2 


<2 ; 
n  (n+1)? n+1 
Equivalently, it suffices to show 
1 1 1 


n+1° (n+1)? “ae 


But we have 
Hs 1 _n+1+l_  n+2 
n+1° (n+1)? (n + 1)? (n+ 1)?” 
Everything in sight is positive, so by clearing denominators, the desired inequality 
is equivalent to 


n? + 2n=n(n4+ 2) < (n41)? =n? 4 2n4+1, 


which is true! Thus we have all the ingredients of an induction proof, but again we 
need to put things together in proper order, a task which we leave to the reader. 


Remark: Taking limits as n — oo, it follows that eee + < 2. In particular, 
this argument shows that the infinite series converges. The exact value of the sum 


is, in fact, = A proof of this requires techniques from advanced calculus. 


5. Extending Binary Properties to n-ary Properties 
Example: All horses have the same color. 
Proposed proof: There are only finitely many horses in the world, so it will suffice 


to show that for all n € Zt, P(n) holds, where P(n) is the statement that in any 
set of n horses, all of them have the same color. 


Base case: In any set S of one horse, all of the horses in S$ have the same color! 
Induction step: We suppose that for some positive integer n, in any set of n horses, 
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all have the same color. Consider now a set of n + 1 horses, which for specificity 
we label Hi, Ho,...,Hn, Hn4i. Now we can split this into two sets of n horses: 


ioe: ee: 


and 
T= {fo, a8 »)An, An +1}. 


By induction, every horse in S has the same color as H;: in particular H,, has 
the same color as H,. Similarly, every horse in T has the same color as Hy: in 
particular H,+1 has the same color as Hy. But this means that Ho,..., Hn, An+1 
all have the same color as Hy. It follows by induction that for all n € Z*, in any 
set of n horses, all have the same color. 


PROOF ANALYSIS: Naturally one suspects that there is a mistake somewhere, and 
there is. However it is subtle, and occurs in a perhaps unexpected place. In fact the 
argument is completely correct, except the induction step is not valid when n = 1: 
in this case S = {H,} and T = {Hz} and these two sets are disjoint: they have 
no horses in common. We have been misled by the “dot dot dot” notation which 
suggests, erroneously, that S and T must have more than one element. 

In fact, if only we could establish the argument for n = 2, then the proof goes 
through just fine. For instance, the result can be fixed as follows: if in a finite set 
of horses, any two have the same color, then they all have the same color. 


There is a moral here: one should pay especially close attention to the smallest 
values of n to make sure that the argument has no gaps. On the other hand, 
there is a certain type of induction proof for which the n = 2 case is the most 
important (often it is also the base case, but not always), and the induction step 
is easy to show, but uses once again the n = 2 case. Here are some examples of this. 


The following is a fundamental fact of number theory, called Euclid’s Lemma. 
PROPOSITION 2.8. Let p be a prime, and leta,b€ Z*. Ifp| ab, p| a orp | b. 


Later in this chapter we will give a proof (yes, by induction!). Let’s assume it for 
now. Then we can swiftly deduce the following useful generalization. 


PROPOSITION 2.9. Let p be a prime number, n € Z* and ay,...,dn € Zt. If 
p|a1-++Gn, then p | a; for somel<i<n. 


ProoF. This is trivial for n = 1. We show it for all n > 2 by induction. 
Base case: n = 2: This is precisely Euclid’s Lemma. 
Induction Step: We assume that for a given n € Zt and ay,...,@n € Z*, if'a prime 
p divides the product a1 ---a,, then it divides at least one a;. Let a1,...,Qn,@n41 € 
Z, and that a prime p divides a1 -+-@n@n41. Then p | (a1 +++ Gn)an41, 80 by Euclid’s 
Lemma, p | a1---@n or p | dn41. If the latter, we’re done. If the former, then by 
our inductive hypothesis, p | a; for some 1 <i < n, so we are also done. 


COROLLARY 2.10. Let p be a prime, and leta,n€ Z*. Thenp|a" => pla. 


EXERCISE 2.2. Use Corollary 2.10 to show that for any prime p, pr is irra- 
tional. 
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6. The Principle of Strong/Complete Induction 


Problem: A sequence is defined recursively by aj = 1, ag = 2 and an = 3an_1 — 
2an—2. Find a general formula for a, and prove it by induction. 


Proof analysis: Unless we know something better, we may as well examine the 
first few terms of the sequence and hope that a pattern jumps out at us. We have 


a3 = 3a2g — 2a, = 3-2-2-1=4. 
a4 = 3a3 — 2a2 = 3-4—-2-2=8. 
a5 = 3a4 — 2a3 = 3-8—-2-4=16. 
ag = 3a5 — 2a4 = 3-16 —-—2-8 = 82. 


The evident guess is therefore a, = 2”~'. Now a key point: it is not possible to 
prove this formula using the version of mathematical induction we currently have. 
Indeed, let’s try: assume that a, = 2”~!. Then 


An+1 = 34n — 2An-1. 
By the induction hypothesis we can replace a, with 2”~1, getting 
An4+1 = 3° on = 2an—13 


now what?? A little bit of thought indicates that we think a,-1 = Qr-2_ Tf for 
some reason it were logically permissible to make that substitution, then we’d be 
in good shape: 


Qn41 =3- gr-l_g.9gn-2 _ 3. 9n-1_9n-1_9.9n-1 _ 9 _ g(n+1)—1 


which is what we wanted to show. Evidently this goes a bit beyond the type of 
induction we have seen so far: in addition to assuming the truth of a statement 
P(n) and using it to prove P(n+ 1), we also want to assume the truth of P(n—1). 


There is a version of induction that allows this, and more: 


Principle of Strong/Complete Induction: 

Let P(n) be a sentence with domain the positive integers. Suppose: 

(i) P(1) is true, and 

(ii) For alln € Zt, if P(1),...,P(n —1), P(m) are all true, then P(n + 1) is true. 
Then P(n) is true for all n € Z*. 


Thus, in a nutshell, strong/complete induction allows us to assume not only the 
truth of our statement for a single value of n in order to prove it for the next value 
n+ 1, but rather allows us to assume the truth of the statement for all positive 
integer values less than n + 1 in order to prove it for n+ 1. 


It is easy to see that PS/CI implies the usual principle of mathematical induc- 
tion. The logical form of this is simply? 


(A = C) => (AAB = C). 


3The symbol A denotes logical conjunction: in other words, A / B means “A and B”. 
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In other words, if one can deduce statement C' from statement A, then one can 
also deduce statement C from A together with some additional hypothesis or hy- 
potheses B. Specifically, we can take A to be P(n), C to be P(n+ 1) and B to be 
PII) AP(Q)A...A P(n—1). 


Less obviously, one can use our previous PMI to prove PS/CI. The proof is not 
hard but slightly tricky. Suppose we know PMI and wish to prove PS/CI. Let P(n) 
be a sentence with domain the positive integers and satisfying (i) and (ii) above. 
We wish to show that P(n) holds for all n € Z*, using only ordinary induction. 
The trick is to introduce a new predicate Q(n), namely 


Q(n) = P(1) A P(2) A... A P(n). 


Notice that Q(1) = P(1); (ii) above tells us that Q(n) = > P(n+ 1). But if we 
know Q(n) = P(1)A...AP(n) and also P(n+1), then we know P(1)A...A P(n)A 
P(n+1) = Q(n+1). So Q(1) holds and for all n, Q(n) = Q(n+1). So by PMI, 
Q(n) holds for all n, hence certainly P(n) holds for all n. 


EXERCISE 2.3. As for ordinary induction, there is a variant of strong/complete 
induction where instead of starting at 1 we start at any integer No. State this 
explicitly. 


Here is an application which makes full use of the “strength” of PS/CI. 


PROPOSITION 2.11. Let n> 1 be an integer. Then there exist prime numbers 
Pi,---,Pk (for some k > 1) such that n = py +--+ px. 


PROOF. We go by strong induction on n. 
Base case: n = 2. Indeed 2 is prime, so we’re good. 
Induction step: Let n > 2 be any integer and assume that the statement is true for 
all integers 2 << k <n. We wish to show that it is true for n. 
Case 1: n is prime. As above, we’re good. 
Case 2: n is not prime. By definition, this means that there exist integers a, b, with 
1<a,b <n, such that n = ab. But now our induction hypothesis applies to both 
a and b: we can write a = p,--- py and b= q,---qm, where the p;’s and q;’s are all 
prime numbers. But then 


n=ab=pi-- pra 


is an expression of n as a product of prime numbers: done! 


This is a good example of the use of induction (of one kind or another) to give a 
very clean proof of a result whose truth was not really in doubt but for which a 
more straightforward proof is wordier and messier. 


7. Solving Homogeneous Linear Recurrences 


Recall our motivating problem for PS/CI: we were given a sequence defined by 
a, = 1, ag = 2, and for all n > 1, ayn = 8an_1 — 2an_2. By trial and error we 
guessed that a, = 2"—', and this was easily confirmed using PS/CI. 


But this was very lucky (or worse: the example was constructed so as to be easy 
to solve). In general, it might not be so obvious what the answer is, and as above, 
this is induction’s Kryptonite: it has no help to offer in guessing the answer. 
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Example: Suppose a sequence is defined by % = 2, &) = 5%,_1 — 3 for alln > 1. 


Here the first few terms of the sequence are x1 = 7, 42 = 32, x3 = 157, x4 = 782, 
“5 = 3907. What’s the pattern? At least to me, it’s not evident. 


This is a case where more generality brings clarity: it is often easier to detect 
a pattern involving algebraic expressions than a pattern involving integers. So 
suppose we have a,b,c € R, and we define a sequence recursively by 


to=c; Vn EN, tn41 = at, 4+ 0. 
Now let’s try again: 
t=at%y +b=ac+b. 
rq = ar, +b=a(ac+b) +b=ca* + ba +b. 
£3 = arg +b = a(ca? + bat b) +b = ca® + ba? + bat b. 
rq = ar3 +b =a(ca? + ba? + ba +b) +b = ca* + ba? + ba? + ba + b. 


Aha: it seems that we have for all n > 1. 


tn = ca” +ba" +. + ba +b. 


Now we have something that induction can help us with: it is true for n = 1. 
Assuming it is true for n, we calculate 


Cn41 = AL, +b = a(ca”+ba"! +... +ba+b)+b) = ca"t!+ba"+---+ba? +ba+b, 


so the desired expression is correct for all n. Indeed, we can simplify it: 


iA n+] _4 b _ nb 
by = aa" +BY a = ca" 40 (4 j=" +ac—c)a 
i=1 


a-—l a-—1l 


In particular the sequence x,, grows exponentially in n. 


Let us try our hand on a sequence defined by a two-term recurrence: 
FP, = Fy =1; Vn > 1, Fase = Fr4i ct Fh. 
The F,,’s are the famous Fibonacci numbers. Again we list some values: 
F3 =2, Fy =3, F5 =5, Fe =8, Fy = 18, Fg = 21, Fo = 34, Figo = 55, 
Fy, = 89, Fig = 144, Fig = 233, Fig = 377, Fis = 377, 
F909 = 280571172992510140037611932413038677189525, 
F991 = 453973694165307953197296969697410619233826. 


These computations suggest F,, grows exponentially. Taking ratios of successive 
values suggests that the base of the exponential lies between 1 and 2, e.g. 


Cognoscenti may recognize this as the decimal expansion of the golden ratio 


1+ 75 
= 
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However, let’s consider a more general problem and make a vaguer guess. Namely, 
for real numbers b,c we consider an recurrence of the form 


(6) 2, = Ay, 2 = Ao, Vn > 1,an42 = bin41 + Cy. 


In all the cases we’ve looked at the solution was (roughly) exponential. So let’s 
guess an exponential solution x, = Cr” and plug this into the recurrence; we get 


Ce = ag = Gn er’), 
which simplifies to 
r? — br —cr =0. 


Evidently the solutions to this are 


b+ Vb? + 4c 
5 : 


\ 


Some cases to be concerned about are the case c = in which case we have 


—b2 
4? 
: ap 
only a single root r = 8, and the case c < — 


oe in which case the roots are complex 
numbers. But for the moment let’s look at the Fibonacci case: b = c = 1. Then 


= ve So we recover the golden ratio yp = 1ivs —a good sign! — as well as 


1-5 
2 

So we have two different bases — what do we do with that? A little thought shows 
that if rj? and r} are both solutions to the recurrence %,42 = b%n41C%n (with any 
initial conditions), then so is Cir} + Car} for any constants C; and C2. Therefore 
we propose t, = Cyr} + Cor} as the general solution to the two-term homoge- 
neous linear recurrence (6) and the two initial conditions 71 = A, x2 = Ag provide 
just enough information to solve for C; and C2. 


= 1— y = —.618033988749894848204586834.... 


Trying this for the Fibonacci sequence, we get 


1 = Fy = Ci(y)’ + C2(1 — 9)”. 
Multiplying the first equation by y and subtracting it from the second equation 
will give us a linear equation to solve for C2, and then we plug the solution into 
either of the equations and solve for C;. It turns out that 
1 —1 
Cy = = 
INTERLUDE: This is easily said and indeed involves only high school algebra. But 


we can do something slicker. Instead of determining the constants by evaluating 
F, at n= 1 and n = 2, it’s easier to evaluate at n = 1 and n = 0: then we have 


Fo = Cry? + Co(1 = yp)? = C7} + Co. 


But for this to work we need to know Fo, which we have not defined. Can it be 
defined in a sensible way? Yes! Writing the basic recurrence in the form F,41 = 
F,, + F,-1 and solving for F;,_1 gives: 


Fy-1 = Fas — Fh. 
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This allows us to define F;, for all integers n. In particular, we have 
fo = Fy-F, =1-1=0. 
Thus we get 
0=C,4+Co, 
whereas plugging in n = 1 gives 
1= Ci(y) + C21 — y) = Ci(y) — C11 — ¢) = (29 - 1)C1, 
1 1 1 -1 
Cy = = = , C= : 
2-1" o(eB)—1 V8 


Now we are ready to prove the following result. 


THEOREM 2.12. (Binet’s Formula) For any n € Z, the nth Fibonacci number 
as 
Fa=e(e"- (1-9) 
n V5 YQ Y > 


14+V5 


where p = 5 


PROOF. We go by strong/complete induction on n. The base cases are n = 1 
and n = 2, but we have already checked these: we used them to determine the 
constants C; and Cy. So now assume that n > 3 and that the formula is correct 
for all positive integers smaller than n+ 2. Then, using the identities 


gy=ptl, 
(1 = ~) = -y" 
1-gt=9?=(-~), 
we compute 
1 n n n n 
Frio = n+l + EF, = el? Y (1 ~) ve (1 Y) )) 


= vy Pied Lp le 


lore") (-~)-"((-¢)" +1) 
= Lyn? -(-9)-"(-9)) = Kr? (9) ) = Ky? -(1-y)"?) 
B B i 


EXERCISE 2.4. Find alln € Z such that F, < 0. 


By the way, it is not quite true that any solution to (6) must have exponential 
growth. For instance, consider the recurrence 


%1=1, 2 = 2; Vn > 1, 8n42 = 2n41 — Vn. 
Then 
vg = 2%. -4, = 2-2-1=8, tq = 243 —-%2 =2-3-2=4, % =2-4-3=5. 


8. THE WELL-ORDERING PRINCIPLE 39 


It certainly looks as though x, =n for all n. Indeed, assuming it to be true for all 
positive integers smaller than n + 2, we easily check 


Un42 = 2p — In = 2A(n4+1)-—n=2n4+2-n=n+2. 


The characteristic polynomial is r? — 2r +1 = (r—1)?: it has repeated roots. One 
solution is Ci1" = C, (i.e., 2, is a constant sequence). This occurs iff x2 = 21, so 
clearly there are nonconstant solutions as well. It turns out that in general, if the 
characteristic polynomial is (x — r)?, then the two basic solutions are 7, = r” and 
also z, = nr”. It is unfortunately harder to guess this in advance, but it is not hard 
to check that this gives a solution to a recurrence of the form %p42 = 2ro@n41 —r2on. 


These considerations will be eerily familiar to the reader who has studied differen- 
tial equations. For a more systematic exposition on “discrete analogues” of calculus 
concepts (with applications to the determination of power sums as in §3), see [DC]. 


8. The Well-Ordering Principle 


There is yet another form of mathematical induction that can be used to give what 
is, arguably, an even more elegant proof of Proposition 2.11. 


THEOREM 2.13. (Well-Ordering Principle) Let S be a nonempty subset of Z*. 
Then S has a least element, i.e., there exists s € S such that for allt ES, 5 <t. 


Intutitively, the statement is true by the following reasoning: first we ask: is 1 € S? 
If so, it is certainly the least element of S. If not, we ask: is 2 € S? If so, it is 
certainly the least element of S. And then we continue in this way: if we eventually 
get a “yes” answer then we have found our least element. But if for every n the 
answer to the question “Is n an element of S?” is negative, then S is empty! 


The well-ordering principle (henceforth WOP) is often useful in its contraposi- 
tive form: if a subset S Cc Z*+ does not have a least element, then S = ©. 


We CLAIM WOP is logically equivalent to the principle of mathematical induc- 
tion (PMI) and thus also to the principle of strong/complete induction (P5/CI). 


First we will assume PS/CI and show that WOP follows. For this, observe that 
WOP holds iff P(m) holds for all n € Zt, where P(n) is the following statement: 


P(n): If SC Zt andn€ S, then S has a least element. 


Indeed, if P(n) holds for all n and S C Z is nonempty, then it contains some 
positive integer n, and then we can apply P(n) to see that S has a least element. 
Now we can prove that P(n) holds for all n by complete induction: first, if 1 € S, 
then indeed 1 is the least element of S, so P(1) is certainly true. Now assume P(k) 
for all 1 < k <n, and suppose that n+1€S. Ifn-+ 1 is the least element of S, 
then we’re done. If it isn’t, then it means that there exists k € S,1<k< S. Since 
we have assumed P(k) is true, therefore there exists a least element of S. 


Conversely, let us assume WOP and prove PMI. Namely, let S C Z and suppose 
that 1 € S, and that for all n, ifn € S then n+ 1 € S. We wish to show that 
S = Z*. Equivalently, putting T = Z* \ S, we wish to show that T = 0. If not, 
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then by WOP T has a least element, say n. Reasoning this out gives an immediate 
contradiction: first, n ¢ S. By assumption, 1 € S$, so we must have n > 1, so that 
we can write n = m+ 1 for some m € Zt. Further, since n is the least element of 
T we must have n — 1 =m € S, but now our inductive assumption implies that 
n+1=n€ S, contradiction. 


So now we have shown that PMI <— > PS/CI = WOP => PMI. 


Let us give another proof of Proposition 2.11 using WOP. We wish to show that 
every integer n > 1 can be factored into primes. Let S be the set of integers n > 1 
which cannot be factored into primes. Seeking a contradiction, we assume S' is 
nonempty. In that case, by WOP it has a least element, say n. Now n is certainly 
not prime, since otherwise it can be factored into primes. So we must have n = ab 
with 1 < a,b < n. But now, since a and 6b are integers greater than 1 which are 
smaller than the least element of S, they must each have prime factorizations, say 
a@=p1-::pr, b=qM--:q. But then (stop me if you’ve heard this one before) 


n= ab= pie Pegs 


itself can be expressed as a product of primes, contradicting our assumption. there- 
fore S is empty: every integer greater than 1 is a product of primes. 


This kind of argument is often called proof by minimum counterexample. 


These two proofs of Proposition 2.11 are very close: the difference between a proof 
by PS/CI and a proof by WOP is more a matter of taste than technique. 


9. The Fundamental Theorem of Arithmetic 


9.1. Euclid’s Lemma and the Fundamental Theorem of Arithmetic. 


The following are the two most important theorems in beginning number theory. 


THEOREM 2.14. (Euclid’s Lemma) Let p be a prime number and a,b be positive 
Suppose that p| ab. Then p|a orp | b. 


THEOREM 2.15. (Fundamental Theorem of Arithmetic) The factorization of 
any integer n > 1 into primes is unique, up to the order of the factors: suppose 


N= Div Pe= Hs 


are two factorizations of n into primes, with py <...< pr andqa <...<q@. Then 
k=landp;=q for dl1<i<k. 


A prime factorization n = pi ---px is in standard form if p; <... < pr. Every 
prime factorization can be put in standard form by ordering the primes from least 
to greatest. Dealing with standard form factorizations is a convenient bookkeeping 
device, since otherwise our uniqueness statement would have to include the proviso 
“up to the order of the factors.” 


Given Proposition 2.11 — ie., the existence of prime factorizations — Theorems 
2.14 and 15.13 are equivalent: each can be easily deduced from the other. 
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EL implies FTA: Assume Euclid’s Lemma. As seen above, this implies Propo- 
sition 2.9: if a prime divides any finite product of integers it must divide one of 
the factors. Our proof will be by minimal counterexample: suppose that there are 
some integers greater than one which factor into primes in more than one way, and 
let n be the least such integer, so 


(7) M=PiPR= NM" Hs 
where each of the primes is written in nonincreasing order. Evidently p; | n = 
g1°-:M, so by Proposition 2.9, we must have that p; | q; for some 1 <j <1. But 
since q; is also prime, this means that p; = q;. Therefore we can cancel them from 
the expression, getting 

n 
(8) Bo eee a ae 
But = is less than the least integer which has two different factorizations into 
primes, so it must have a unique factorization into primes, meaning that the primes 
on the left hand side of (8) are equal, in order, to the primes on the right hand 
side of (8). This also implies that p; = q,; is less than or equal to all the primes 
appearing on the right hand side, so 7 = 1. Thus k = 1, pj = q; =m and p; = q; 
for 2<%i<j. This means that in (7) the two factorizations are the same after all! 


FTA implies EL: Assume every integer greater than one factors uniquely into 
a product of primes, let p be a prime, and let a and b be positive integers such that 
p | ab. If either a or b is 1, then the other is just p and the conclusion is clear, so 
we may assume a,b > 1 and therefore have unique prime factorizations 


a=pr Pr, D=Q1°°* Gs; 


our assumption that p divides ab means ab = kp for some k € Zt and thus 


ab = pi +++ PrGi+++ ds = kp. 
The right hand side of this equation shows that p must appear in the prime factor- 


ization of ab. Since the prime factorization is unique, we must have at least one p; or 
at least one q; equal to p. In the first case p divides a; in the second case p divides b. 


The traditional route to FTA is via Euclid’s Lemma, and the traditional route 
to Euclid’s Lemma (Euclid’s route in Elements) is via a series of intermediate steps 
including the Euclidean algorithm and finding the set of all integer solutions to 
equations of the form az+by = 1. This takes some time to develop — perhaps a week 
in an elementary number theory course. But we can bypass all these intermediate 
steps and give direct inductive proofs of both EL and FTA. And we will. 


9.2. Rogers’ Inductive Proof of Euclid’s Lemma. 


Here is a proof of Euclid’s Lemma using WOP, following K. Rogers [R063]. 
Seeking a contradiction, we suppose there is at least one prime such that Eu- 

clid’s Lemma does not hold for that prime. By WOP there is a least such prime, 

say p, so there are a,b € Z* with p | ab but p{a and pf b. By WOP there is a 

least a € Z* such that there is at least one b € Z* with p | ab and p{a, pt b. 
Now consider the following equation: 


ab = (a— p)b+ pb, 
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which shows that p | ab <=> p|(a—p)b. There are three cases: 

Case 1: a —p is a positive integer. Then, since 0 < a—p < a and a was by 
assumption the least positive integer such that Euclid’s Lemma fails for the prime 
p, we must have that p | a—p or p| b. By assumption p { b, so we must have 
p|a-—p, but then p | (a — p) + p =a, contradiction! 

Case 2: We have a = p. But then p | a, contradiction. 

Case 3: We have a < p. On the other hand, a > 1 — if p|1-b, then p | b— so by 
Proposition 2.11 a is divisible by some prime gq, and q | a < p, so g < p. Therefore q 
is a prime which is smaller than the least prime for which Euclid’s Lemma fails, so 
Euclid’s Lemma holds for g. Since p | ab, we may write pk = ab for some k € Zt, 
and now q|a = > q|ab= pk, so by Euclid’s Lemma for q, g | p or g | k. The first 
case is impossible since p is prime and 1 < q < p, so we must have q | k. Therefore 


D (4) a (2) b, so p | a0: But 1 < a < a and a is the least positive integer for 
which Euclid’s Lemma fails for p and a, so it must be that p | a (so in particular 


p | a) or p | b. Contradiction. So Euclid’s Lemma holds for all primes p. 
9.3. The Lindemann-Zermelo Inductive Proof of FTA. 


Here is a proof of FTA using WOP, following Lindemann [Li33] and Zermelo [Ze34]. 
We claim that the standard form factorization of a positive integer is unique. 

Assume not; then the set of positive integers which have at least two different 

standard form factorizations is nonempty, so has a least element, say n, where: 


(9) N=Di-- Pp =U" As: 
Here the p;’s and q;’s are prime numbers, not necessarily distinct from each other. 
However, pi # q; for any j. Indeed, if we had such an equality, then after relabelling 
the q;’s we could assume p; = q; and then divide through by p; = q, to get a smaller 
positive integer ae By the assumed minimality of n, the prime factorization of i 
must be unique: i.e., 7 —1=s—1 and p; = q for all 2 <<i<r. Then multiplying 
by pi = q we see that we didn’t have two different factorizations after all. 

In particular p; 4 q. Without loss of generality, assume p; < q. Then, if we 
subtract piq2-+:qs from both sides of (9), we get 


(10) M = 1 — piga-++ ds = Pi(P2°-* Pr — 42°+-Gs) = (G1 — P1)(G2-- Gs). 

Evidently 0 < m < n, so by minimality of n, the prime factorization of m must 
be unique. However, (10) gives two different factorizations of m, and we can use 
these to get a contradiction. Specifically, m = pi(p2--+p, — go+++ Gs) shows that 
pi | m. Therefore, when we factor m = (q — p1)(q2---qs) into primes, at least 
one of the prime factors must be p;. But q2,...,q; are already primes which are 
different from pi, so the only way we could get a p; factor is if pi | (qi — pi). But 
this implies p; | qi, and since q; is also prime this implies p; = q. Contradiction! 


9.4. A Generalized Euclid’s Lemma. 


THEOREM 2.16. (Generalized Euclid’s Lemma) Let a,b,c € Z*. Suppose that 
x|yz and x and y are relatively prime. Then x | z. 


EXERCISE 2.5. a) Prove Theorem 2.16 using the Fundamental Theorem of 
Arithmetic. 
b) Explain why Theorem 2.16 is indeed a generalization of Euclid’s Lemma. 


CHAPTER 3 


Polynomial and Rational Functions 


1. Polynomial Functions 


Using the basic operations of addition, subtraction, multiplication, division and 
composition of functions, we can combine very simple functions to build large and 
interesting (and useful!) classes of functions. For us, the two simplest kinds of 
functions are the following: 


Constant functions: for each a € R there is a function Cg : R > R such that for 
all « € R, Ca(x) =a. In other words, the output of the function does not depend 
on the input: whatever we put in, the same value a will come out. The graph of 
such a function is the horizontal line y = a. Such functions are called constant. 


The identity function J : R — R by I(x) = x. The graph of the identity 
function is the straight line y = z. 


Recall that the identity function is so-called because it is an identity element for 
the operation of function composition: that is, for any function f : R > R we have 


Tof=fol=f. 


Example: Let m,b € R, and consider the function ZL: R > R by x ma-+ b. 
Then LF is built up out of constant functions and the identity function by addition 
and multiplication: LD = Ci, -I+ Cy. 


Example: Let n € Zt. The function m, : 2 1 2” is built up out of the iden- 
tity function by repreated multiplication: m, = I-I---I (n I’s altogether). 


The general name for a function f : R — R which is built up out of the identity 
function and the constant functions by finitely many additions and multiplications 
is a polynomial. In other words, every polynomial function is of the form 


(11) fitra,r”+...+a1% + a9 


for some constants ag,...,@n € R. 


However, we also want to take — at least until we prove it doesn’t make a dif- 
ference — a more algebraic approach to polynomials. Let us define a polynomial 
expression as an expression of the form ae a,x’. Thus, to give a polynomial 
expression we need to give for each natural number 7 a constant a;, while requiring 
that all but finitely many of these constants are equal to zero: i.e., there exists 
some n € N such that a; = 0 for all i > n. 
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Then every polynomial expression f = )7"_9 a,;x' determines a polynomial func- 
tion «++ f(x). But it is at least conceivable that two different-looking polynomial 
expressions give rise to the same function. To give some rough idea of what I mean 
here, consider the two expressions f = 2arcsinz + 2arccosxz and g = z. Now it 
turns out for all « € [—1,1] (the common domain of the arcsin and arccos func- 
tions) we have f(x) = 7. (The angle @ whose sine is x is complementary to the 
angle py whose cosine is x, so arcsinz + arccosz = 6+ y = 5.) But still f and g are 
given by different “expressions”: if I ask you what the coefficient of arcsin x is in 
the expression f, you will immediately tell me it is 2. If I ask you what the coeffi- 
cient of arcsin x is in the expression 7, you will have no idea what I’m talking about. 


One special polynomial expression is the zero polynomial. This is the poly- 
nomial whose ith coefficient a; is equal to zero for all ¢ > 0. 


Every nonzero polynomial expression has a degree, which is a natural number, 
the largest natural number 7 such that the coefficient a; of x’ is nonzero. Thus in 
(11) the degree of f is n if and only if a, 4 0: otherwise the degree is smaller than n. 


Although the zero polynomial expression does not have any natural number as 
a degree, it is extremely convenient to regard deg 0 as negative, i-e., such that deg 0 
is smaller than the degree of any nonzero polynomial. This means that for any 
déN “the set of polynomials of degree at most d” includes the zero polynomial. 
We will follow this convention here but try not to make too big a deal of it. 


Let us give some examples to solidify this important concept: 


The polynomials of degree at most 0 are the expressions f = ao. The corresponding 
functions are all constant functions: their graphs are horizontal lines. (The graph 
of the zero polynomial is still a horizontal line, y = 0, so it is useful to include the 
zero polynomial as having “degree at most zero”.) 


The polynomials of degree at most one are the linear expressions L = mz + b. 
The corresponding functions are linear functions: their graphs are straight lines. 
The degree of L(x) is one if m # 0 — ie., if the line is not horizontal — and 0 if 
m=0Oandb#0. 


Similarly the polynomials of degree at most two are the quadratic expressions 
q(x) = ax? +bc+c. The degree of g is 2 unless a = 0. 
We often denote the degree of the polynomial expression f by deg f. 


THEOREM 3.1. Let f,g be nonzero polynomial expressions. 


a) If f +g #0, then deg(f + g) < max(deg f, deg 9). 
b) We have deg(fg) = deg f + deg g. 


PROOF. a) Suppose that 


f(a) =amu" +... +414 +40, Gm #0 


and 


g(x) = bnpa” +... +b1:4 + bo, bn #0 
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so that degg =m, degg =n. 

Case 1: m > n. Then when we add f and g, the highest order term will be a2, 
since the polynomial g only smaller powers of x. In particular the degree of f +g 
is m = max(m,n). 

Case 2: m <n. Similarly, when we add f and g, the highest order term will be 
Anz", so the degree of f +g is n = max(m,n). 

Case 3: Suppose m = n. Then 


(f +.9)(2) = (Gm + Om)u”™ +... + (a1 + b1)@ + (ao + bo). 
Thus the degree of f + g is at most m. It will be exactly m unless am + bm = 0, 
i.e., unless b,, = —Gm; in this case it will be strictly smaller than m. 


b) If f and g are as above, then the leading term of f -g will be amb,2™t”, and 
since Am, bn #0, Gmbn #0. Thus deg fg =m+n. 


EXERCISE 3.1. For polynomial expressions f = >7)".) a;x* and g = ae, bj x, 
define 


(fo g)(x) = So ai(S byx?)”. 


i=0 = j=0 
Show that deg(f og) = (deg f) (deg g). 


The following is the most important algebraic property of polynomials. 


THEOREM 3.2. (Polynomial Division With Remainder) Let a(x) be a polyno- 
mial expression and b(x) be a nonzero polynomial expression. There are unique 
polynomial expressions q(x) and r(x) such that 
(i) a(x) = a(2)B(c) + r(x) and 
(ti) deg r(x) < deg (x). 


PRoor. First note that we really get to choose only q(x), for then r(x) = 

a(x) — q(z)b(x). Also let us denote the leading coefficient of a(x) by a and the 
leading coefficient of b(x) by 6. Step 1: We prove the existence of q(x) and b(z). 
Case 1: deg a(x) < deg b(x). Take q(x) = 0: deg r(x) = deg a(x) < deg b(z). 
Case 2: dega(x) > degb(x). Take q(x) = Ga a> areal) The point of this 
is that qi(x)b(x) has degee deg a(x) — deg b(x) + deg b(x) = deg a(x) and leading 
coefficient $ - 8 = a, so in r(x) = a(x) — qi(x)b(x) the leading terms cancel and 
deg r1(x) < deg(a(x))—1. If degri(x) < deg (x) then we’re done, take q(x) = q(x) 
and r(x) =1;(x). On the other hand, if degri(x) > deg b(x) we apply the process 
again with ri(x) in place of a(x): let ai be the leading coefficient of ri(x) and 
take go(x) = ao ri(@)—deg (@) so that in re(a) = r1(x) — q2(x)b(x) the leading 
terms cancel to give degra(x) < deg(ri(x)) — 1 < deg(a(x)) — 2. Continue in 
this way generating polynomial expressions q;(x) and r;(x) until we reach a k with 
deg rz, (x) < deg b(x). Then: 


rp (©) = Te-1(@) — qu(x)b(z) 
= Tp—2(@) — qr—1(#) d(x) — qeb(a) 
= rp—3(L) — qp—2(x)b(@) — qu—1(%)b(@) — qub(x) 
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Thus we may take q(x) = qi(x)+...+¢94(2), so that r(x) = a(x)—q(x)b(x) = rz (x) 
and deg r(a) = deg rz(x) < deg b(x). 
Step 2: We prove the uniqueness of q(x) (and thus of r(x) = a(x) — q(x)b(a)). 
Suppose @(x) is another polynomial such that R(x) = a(x) — Q(x)b(x) has degree 
less than the degree of b(x). Then 

a(x) = g(x)b(x) + r(x) = Q(#)b(z) + R(a), 
sO 

(q(x) — Q(2))b(a) = R(x) — r(a). 

Since r(#) and R(x) both have degree less than deg b(x), so does r(x) — R(x), so 
deg b(x) > deg(R(x)—r(x)) = deg((q(@)—Q(2))b(a)) = deg(q(x) —Q(x))+deg (x). 
Thus deg(q(x) — Q(x)) < 0, and the only polynomial with negative degree is the 
zero polynomial, i.e., g(x) = Q(x) and thus r(x) = R(z).! 


EXERCISE 3.2. Convince yourself that the proof of Step 1 is really a careful, 
abstract description of the standard high school procedure for long division of poly- 
nomials. 


Theorem 3.2 has many important and useful consequences; here are some of them. 


THEOREM 3.3. (Root-Factor Theorem) Let f(x) be a polynomial expression and 
ca real number. The following are equivalent: 


(i) f(c) =90. (‘c is a root of f.”) 
(ii) There is some polynomial expression q such that as polynomial expressions, 
f(x) = (a@—c)q(a). (“cx —c is a factor of f.”) 

PRoor. We apply the Division Theorem with a(x) = f(x) and b(a) = x —<¢, 
getting polynomials g(x) and r(a) such that 

f(x) = (@— e)q(@) + r(x) 

and r(a) is either the zero polynomial or has degr < dega —c = 1. In other 
words, r(x) is in all cases a constant polynomial (perhaps constantly zero), and its 
constant value can be determined by plugging in x = c: 


fo) = (e— o)a(e) + re) = r(O). 
The converse is easier: if f(x) = (x — c)q(x), then f(c) = (ec — c)q(c) = 0. 


COROLLARY 3.4. Let f be a nonzero polynomial of degree n. Then the corre- 
sponding polynomial function f has at most n real roots: i.e., there are at most n 
real numbers a such that f(a) =0. 


PROoF. By induction on n. 
Base case (n = 0): If deg f(x) = 0, then f is a nonzero constant, so has no roots. 
Induction Step: Let n € N, suppose that every polynomial of degree n has at 
most n real roots, and let f(x) be a polynomial of degree n+ 1. If f(a) has no 
real root, great. Otherwise, there exists a € R such that f(a) = 0, and by the 
Root-Factor Theorem we may write f(x) = (a — a)g(x). Moreover by Theorem 
3.1, we have n+ 1 = deg f = deg(a — a)g(x) = deg(x — a) + degg = 1 + degg, so 


lig you don’t like the convention that the zero polynomial has negative degree, then you 
can phrase the argument as follows: if q(x) — Q(x) were a nonzero polynomial, these degree 
considerations would give the absurd conclusion deg(q(x) — Q(x)) < 0, so q(x) — Q(x) = 0. 
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deg g = n. Therefore our induction hypothesis applies and g(x) has m distinct real 
roots @1,...,@m for some 0 <m<n. Then f has either m+ 1 real roots — if a is 
distinct from all the roots a; of g — or m real roots — if a = a; for some 7, so it has 
at most m+1<n+1 real roots. 


LEMMA 3.5. Let f = D6 a,x’ be a polynomial expression. Suppose that the 
function f(x) = S7i)aiz* is the zero function: f(x) = 0 for all x € R. Then 
a; =0 for alli, t.e., f is the zero polynomial expression. 


PROOF. Suppose that f is not the zero polynomial, i.e., a; # 0 for some i. 
Then it has a degree n € N, so by Corollary 3.4 there are at most n real numbers c 
such that f(c) = 0. But this is absurd: f(x) is the zero function, so for all (infinitely 
many!) real numbers c we have f(c) = 0. 


THEOREM 3.6. (Uniqueness Theorem For Polynomials) Let 
f =ana" +... +4," + a0, 


g = byt” +...+ 0,4 + bo 
be two polynomial expressions. The following are equivalent: 
(i) f and g are equal as polynomial expressions: for all0 <i <n, a; = 0b. 
(it) f and g are equal as polynomial functions: for allc € R, f(c) = g(c). 
(itt) There are cy < co <... < (n41 such that f(c;) = g(a) for 1 <i<n+l. 


PROOF. (i) => (ii): This is clear, since if a; = b; for all i then f and g are 
being given by the same expression, so they must give the same function. 
(ii) ==> (iii): This is also immediate: since f(c) = g(c) for all real numbers c, we 


may take for instance cj = 1, cg = 2,...,Cng41 =n 1. 
(iii) == (i): Consider the polynomial expression 
h=f —g = (Gn — bn)a” +...+ (a1 — b1)a + (a0 — bo). 


Then h(c1) = f(ce1) — g(c1) = 0,...,h(en41) = flengi) — glen41) = 0. Sohisa 
polynomial of degree at most n which has (at least) n +1 distinct real roots. By 
Corollary 3.4, h must be the zero polynomial expression: that is, for allO <i<n, 
a; — b; = 0. Equivalently, a; = 6; for all 0 < i < n, so f and g are equal as 
polynomial expressions. 


In particular, Theorem 3.6 says that if two polynomials f(a) and g(x) look differ- 
ent —i.e., they are not coefficient-by-coefficient the same expression — then they are 
actually different functions, i.e., there is some c € R such that f(c) 4 g(c). 


Finally, we want to prove an arithmetic result about polynomials, the Rational 
Roots Theorem. For this we need another number-theoretic preliminary. Two 
positive integers a and b are coprime (or relatively prime) if they are not both 
divisible by any integer d > 1 (equivalently, they have no common prime factor). 


THEOREM 3.7. (Rational Roots Theorem) Let ao,...,@n be integers, with ao, an A 
0. Consider the polynomial 


P(x) = anu" +... +41" + a0. 


Suppose that 8 is a rational number, written in lowest terms, which is a root of P: 
P(2) =0. Then ag is divisible by b and a, is divisible by c. 
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PROOF. We know 


b b” b 
0=P(2) =a, +... tay—-+a9. 


c c” c 
Multiplying through by c” clears denominators, giving 
nb” + dnb" !e +... + abe"! + age” = 0. 
Rewriting this equation as 
a,b" = e(—a,_1b""! ws agc”—") 


shows that a,b” is divisible by c. But since b and c have no prime factors in 
common and 6” has the same distinct prime factors as does b, b” and c have no 
prime factors in common and are thus coprime. So Theorem 2.16 applies to show 
that an is divisible by c. Similarly, rewriting the equation as 


aoc” = b(—a,b"? ap ib ee aye") 


shows that aoc” is divisible by b. As above, since b and c are coprime, so are b and 
c”, so by Theorem 2.16 ao is divisible by b. 


In high school algebra the Rational Roots Theorem is often used to generate a finite 
list of possible rational roots of a polynomial with integer coefficients. This is nice, 
but there are more impressive applications. For instance, taking a, = 1 and noting 
that 1 is divisible by c iff c= +1 we get the following result. 


COROLLARY 3.8. Let ao,...,@n—-1 € Z, and consider the polynomial 
P(x) =a" +ay_12" 1+... +012 + ao. 
Suppose c is a rational number such that P(c) =0. Then c is an integer. 
So what? Let p be any prime number, let n > 2, and consider the polynomial 
P(x) =a" —p. 


By Corollary 3.8, if c € Q is such that P(c) = 0, then c € Z. But if c € Z is such 
that P(c) = 0, then c” = p. But this means that the prime number p is divisible 
by the integer c, soc = £1 or c = +p. But (+1)” = +1 and (+p)" = +p”, so 
c” = pis impossible. So there is no c € Q such that c” = p: that is, ;/p is irrational. 


This is a doubly infinite generalization of our first irrationality proof, that 2 
is irrational, but the argument is, if anything, shorter and easier to understand. 
(However, we did use — and prove, by induction — the Fundamental Theorem of 
Arithmetic, a tool which was not available to us at the very beginning of the 
course.) Moral: polynomials can be useful in surprising ways! 


PROPOSITION 3.9. For every polynomial P of positive degree there are irre- 
ducible polynomials p,,...,pe such that 


P=pi--'Dk- 


EXERCISE 3.3. Prove it. (Suggestion: adapt the proof that every integer n > 1 
can be factored into a product of primes.) 
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2. Rational Functions 


A rational function is a function which is a quotient of two polynomial functions: 
f(z) = aa: with Q(x) not the zero polynomial. To be sure, Q is allowed to have 
roots; it is just not allowed to be zero at all points of R. 


The “natural domain” of ate is the set of real numbers for which Q(x) ¥ 0, 


ie., all but a finite (possibly empty) set of points. 


2.1. The Partial Fractions Decomposition. 


The goal of this section is to derive the Partial Fractions Decomposition (PFD) 
for a proper rational function. The PFD is a purely algebraic fact which is useful 
in certain analytic contexts: especially, it is the key technique used to find explicit 
antiderivatives of rational functions. 


THEOREM 3.10. Let P,, P2,P be polynomials, with P, monic irreducible and 

P, not divisible by P,. Then there are polynomials A, B such that 
AP, + BP, =P. 

Proor. Let S be the set of nonzero polynomials of the form aP, + bP» for 
some polynomials a,b. Since P;) = 1-P,+0-Po, Pp =0-P,+1-Py areinS,S#2@. 
By the Well Ordering Principle there is m € S of minimal degree: write 

aP, + bP» =m. 
If a polynomial lies in S then so does every nonzero constant multiple of it, so we 


may assume m is monic. We claim that for any M = AP, + BP, € S, we must 
have m | M. To see this, apply Polynomial Division to M and m, getting 


M=Qm-+ R, deg R < degm. 
Since also 
R=M —-Qm=(A- Qa)P, + (B — Qb)Po, 
so if R 4 0 we’d have R € S of smaller degree than m, contradiction. Thus R = 0 
and m | M. Since P; € S, m | P,. Because P; is monic irreducible, m = P, or 
m =1. Since P2 € S, also m | P2, and so by hypothesis m #4 P;. Thus m = 1 and 


aP, + bP2 =1. 
Multiplying through by P we get 
(aP)P, + (bP) Po = P, 
so we may take A=aP, B = bP. 


COROLLARY 3.11. (Euclid’s Lemma For Polynomials) Let p be an irreducible 
polynomia, and suppose p| P,P2. Then p| P, or p| Po. 


ProoF. As is usual in trying to prove statements of the form “A implies (B or 
C)”, we assume that B is false and show that C is true. Here this means assuming 
p{ P,. By Theorem 3.10, there are polynomials A and B such that 


Ap + BP, el 
Multiplying through by P, gives 
ApP» + BPP» = Py. 
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Since p | ApP, and p| BP, P:, p| (Ap2+ BP, P2) = Pp». 


EXERCISE 3.4. Show that every monic polynomial of positive degree factors 
uniquely into a product of monic irreducible polynomials. 


COROLLARY 3.12. Let n € Z*, and let p, P,Qo be polynomials, such that p is 
irreducible and that p does not divide Qo. 
a) There are polynomials B,A such that we have a rational function identity 


P(t) _ Bix) , A(z) 
a Peer Qo@) play” | pay *Qoa) 
b) If deg P < deg(p"Qo), we may choose A and B such that 


deg A < deg(p”~'Qo), deg B < degp”. 


PRooF. a) Apply Theorem 3.10a) with P, = p, P2 = Qo: we get 
(13) P= Ajp+ BQ. 
for some polynomials A,, B. Thus 
P  — Aip+BQo._ B A, 


Pp" Qo p”Qo pe) POO y: 
b) By Polynomial Division (Theorem 3.2), there are C,r such that B = Cp+r and 
degr < deg p. Substituting in (13) and putting A = Ai +CQo we get 
P= Aip+ BQo = Aip+ (Cp +1)Qo = (A1 + CQo)p + rqo = Ap + rQo. 
We have 
deg(rQo) = degr + deg Qo < degp + deg Qo < deg p” + deg Qo = deg(p" Qo). 
Since also deg P < deg(p"Qo) and Ap = P — rQp, it follows that 


deg A + deg p = deg(Ap) < deg(p"Qo) = deg Qo + ndeg p, 


so 


deg A < deg Qo + (n — 1) deg p = deg(p""'Qo). 


The identity of Corollary 3.12 can be applied repeatedly: suppose we start with a 
proper rational function a with Q a monic polynomial (as is no loss of generality; 
we can absorb the leading coefficient into P). Then Q is a polynomial of positive 


degree, so we may factor it as 


Q= pt Pr, 
where pi,...,p; are distinct monic irreducible polynomials. Let us put Qo = 
ps? +: pe", so Q = pj? Qo. Applying Corollary 3.12, we may write 
Bes Bic s: sees 
Q py . pt Qo 
with deg Ba, < deg p; and deg Ag, < deg(p? 'Qo). Thus Corollary 3.12 applies to 


Aa 


ac 
pit Qo 


, as well, giving us overall 


P = Ba, Bay-1 i Aa,-1 
= T = T a : 
Q pt pt * — pi-7Qo 
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And we may continue in this way until we get 

P_ Ba , Bi , Ao 

Q pt’ peas © Qo’ 
with each deg B. < deg P; and deg Ap < deg Qo. Recalling that Qo = pS? ---p%", 
we may put Qo = p5°Q, and continue on. Finally we get an identity of the form 
P — Bia, _ Bra Bri _ Bray 


| 
ai ea | | om | 


PU Pr py Pi Pr 

with deg B;,; < degp; for all i and j. From an algebraic perspective, (14) is the 

Partial Fractions Decomposition. In fact, though it is not the point for us, 

everything that we have done works for polynomials with coefficients in any field K 
eg. K =Q,K =C, K =Z/pZ. Conversely, when we work with polynomials over 

the real numbers, the expression simplifies, because the classification of irreducible 

polynomials over the real numbers is rather simple. Indeed: 


(14) 


LEMMA 3.13. The irreducible (real!) polynomials are precisely: 
(i) The linear polynomials (a) = a(x — c) for a,c € R, a #0; and 
(ii) The quadratic polynomisls Q(x) = ax? + bx +c for a,b,c € R, b? — 4ac < 0. 


Every linear polynomial is irreducible. Further, from the Rational Roots Theorem, 
for a polynomial of degree greater than 1 to be irreducible, it cannot have any real 
roots. For quadratic (and, for the record, cubic) polynomials this is also sufficient, 
since a nontrivial factorization of a degree 2 or 3 polynomial necessarily has a linear 
factor and thus a real root. It is a a consequence of the Quadratic Formula that a 
real quadratic polynomial Q(x) = ax? + bx +c has no real roots iff its discriminant 
b? — 4dac is negative. Thus half of Lemma 3.13 — the half that says that a linear 
polynomials and quadratic polynomial without real roots are irreducible — is easy. 
The other half of Lemma 3.13 — the half that says that there are no other 
irreducible polynomials — is far from easy: in fact it lies well beyond our current 
means. In several hundred pages’ time we will return to prove this result as a 
consequence of the Fundamental Theorem of Algebra: see Theorem 15.17. 
Combining (14) and Lemma 3.13 we get the desired result. 


THEOREM 3.14. (Real Partial Fractions Decomposition) Let ae be a proper 


rational function, with 

Q(x) = (a — 1) ++ (@ = eK) ™* qu (a) + u(x); 
here cy,...,Ck ER, witha £0, c1,..., cx distinct; and q,...,q are distinct monic 
irreducible quadratics. There are real numbers A..,B..., C.,. such that we have an 


identity of rational functions 


(15) 
P(x) = ma Ali Mr Agi | ni By jt + Ow | , ny Buia Et Gy 
OG) ae Gee 2 bt > : 


22 ail) 22 aie} 


EXERCISE 3.5. Show that the constants in (15) are unique. 


CHAPTER 4 


Continuity and Limits 


1. Remarks on the Early History of the Calculus 


We have seen that in order to define the derivative f’ of a function f : RR > R we 
need to understand the notion of a limit of a function at a point. It turns out that 
giving a mathematically rigorous and workable definition of a limit is hard — really 
hard. Let us begin with a quick historical survey. 


It is generally agreed that calculus was invented (discovered?) independently by 
Isaac Newton and Gottfried Wilhelm von Leibniz, roughly in the 1670’s. Leibniz 
was the first to publish on calculus, in 1685. However Newton probably could have 
published his work on calculus before Leibniz, but held it back for various reasons.? 

To say that “calculus was discovered by Newton and Leibniz” is an oversim- 
plification. Computations of areas and volumes which we can now recognize as 
using calculus concepts go back to ancient Egypt, if not earlier. The Greek math- 
ematicians Eudoxus (408-355 BCE) and Archimedes (287-212 BCE) developed 
the method of exhaustion, a limiting process which anticipates integral calculus. 
Also Chinese and Indian mathematicians made significant achievements. Even in 
“modern” Europe, Newton and Leibniz were not functioning in a complete intel- 
lectual vacuum. They were responding to and continuing earlier work by Pierre 
de Fermat (on tangent lines) and John Wallis, Isaac Barrow and James Gre- 
gory. This should not be surprising: all scientific work builds on work of others. 
But the accomplishments of Newton and Leibniz were so significant that after their 
efforts calculus existed as a systematic body of work, whereas before them it did not. 


How did Newton and Leibniz construe the fundamental concept, namely that of 
a limit? Both of their efforts were far from satisfactory, indeed far from making 
sense. Newton’s limiting concept was based on a notion of fluxions, which is so 
obscure as not to be worth our time to describe it. Leibniz, a philosopher and writer 
as well as a mathematician, addressed the matter more thoroughly and came up 
with the notion of an infinitesimal quantity, a number which is not zero but 
“vanishingly small”, i.e., smaller than any “ordinary” positive number. 


The concept of infinitesimals has been taken up by mathematicians since Leib- 
niz, and eventually with complete mathematical success...but not until the 1960s!” 
For instance one has a definition of an infinitesimal element x of an ordered field 
K, namely an element x which is positive but smaller than 4 for alln € Z*. It is 


Yy highly recommend James Gleick’s biography of Newton. If I wanted to distill hundreds of 
pages of information about his personality into one word, the word I would choose is...weirdo. 
2See http://en.wikipedia.org/wiki/Nonstandard_analysis for an overview of this story. 
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easy to see that an ordered field admits infinitesimal elements iff it does not satisfy 
the Archimedean axiom, whereas the real numbers R do satisfy the Archimedean 
axiom. So at best Lebiniz was advocating a limiting process based on a different 
mathematical model of the real numbers than the “standard” modern one. And 
at worst, Lebiniz’s writing on infinitesimals seems like equivocation: at different 
stages of a calculation the same quantity is at one point “vanishingly small” and at 
another point not. The calculus of both fluxions and infnitesimals required, among 
other things, some goodwill: if you used them as Newton and Leibniz did in their 
calculations then at the end you would get a sensible (in fact, correct!) answer. 
But if you wanted to make trouble and ask why infinitesimals could not be manip- 
ulated in other ways which swiftly led to contradictions, it was all too easy to do so. 


The calculus of Newton and Leibniz had a famous early critic, Bishop George 
Berkeley. In 1734 he published The Analyst, subtitled “A DISCOURSE Addressed 
to an Infidel MATHEMATICIAN. WHEREIN It is examined whether the Object, 
Principles, and Inferences of the modern Analysis are more distinctly conceived, 
or more evidently deduced, than Religious Mysteries and Points of Faith.” Fa- 
mously, Berkeley described fluxions as the ghosts of departed quantities. I haven’t 
read Berkeley’s text, but from what I am told it displays a remarkable amount of 
mathematical sophistication and most of its criticisms are essentially valid! 


So if the mid 17th century is the birth of systematic calculus it is not the birth 
of a satisfactory treatment of the limiting concept. When did this come? More 
than 150 years later! The modern definition of limits via inequalities was given by 
Bolzano in 1817 (but not widely read), in a somewhat imprecise form by Cauchy 
in his influential 1821 text, and then finally by Weierstrass around 1850. 


2. Derivatives Without a Careful Definition of Limits 


EXAMPLE 4.1. Let f(x) = ma +b be a linear function. Then f has the follow- 
ing property: for any x1 # x2, secant line between the two points (x11, f(a1)) and 
(x2, f(a2)) is the line y = f(a). Indeed, the slope of the secant line is 


f(a) — f(z) = maa+b—(ma,+b) — m(x2- 21) - 


TQ — Ly t2— © t2— X41 
Thus the secant line has slope m and passes through the point (x11, f(a1)), as does 
the linear function f. But there is a unique line passing through a given point with 
a given slope, so that the secant line must be y= ma + b. 


Using this, it is now not at all difficult to compute the derivative of a linear func- 
tion...assuming an innocuous fact about limits. 


EXAMPLE 4.2. Let f(a) =ma-+b. Then 


f(z) ina ee) = jim m(@ +h) +b — (ma +b)h = tim = lim m. 
The above computation is no surprise, since we already saw that the slope of any 
secant line to a linear function y = ma + b is just m. So now we need to evaluate 
the limiting slope of the secant lines. But surely if the slope of every secant line 
is m, the desired limiting slope is also m, and thus f’(x) = m (constant function). 
Let us record the fact about limits we used. 
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Fact 4.1. The limit of a constant function f(x) = C as x approaches a is C. 
EXAMPLE 4.3. Let f(x) =a?. Then 
h) — h 2 _ 72 
(2) = tim {2+ =f) _ 5, @+N?=s 


h0 h h—0 h 
24. 9¢h + h? — x? h(2 h 
= lim ~ zee “= lim zag 1 eb Ae 
h>0 h h0 h h—0 


Now Leibniz would argue as follows: in computing the limit, we want to take h 
infinitesimally small. Therefore 2x + h is infinitesimally close to 2x, and so in the 
limit the value is 2x. Thus 
f'(x) = 2a. 

But an equally accurate description of what we have done is as follows: we simplified 
the difference quotient flerh) Me) until we got an expression in which it made 
good sense to plug inh =0, and then we plugged inh = 0. If you wanted to give 
a freshman calculus student practical instructions on how to compute derivatives 
of reasonably simple functions directly from the definition, I think you couldn’t do 
much, better than this! 


EXAMPLE 4.4. Let f(x) =x°. Then 


fl(w) =1 fle+h)—f(@) _,, (e+h)P-2* _,. x? + 37h + 3rh? +h? — 2 
ae h ~ pO h he h 
h(3a? ht+h? 
= lim Eanes) 2 lim 32? + 32h + h?. 
h—-0 h h-0 
Again we have simplified to the point where we may meaningfully set h = 0, getting 
f'(z) = 32”. 


EXAMPLE 4.5. Forn € Zt, let f(x) =x". Then 


Meted at! ETI SS PS oe NO a = 
Fa) = Be hi = Tan h ~ SO h 


ere (Dee al eee NY n-1 “S (n n-ipi-2 
ho h = jim (7) +nyo(i)e i 


At this point we have seen many examples of a pleasant algebraic phenomenon. 
Namely, for y = f(x) a polynomial function, when we compute the difference 
quotient eth)= He) we find that the numerator, say G(h) = f(a +h) — f(a), 
always has h as a factor: thus we can write it as G(h) = hg(h), where g(h) is 
another polynomial in h. This is exactly what we need in order to compute the 
derivative, because when this happens we get 
a+h)— f(a hg(h 
Fe) = fi EERE J SP fi 00) = 9) 

Can we guarantee that this factorization f(a +h) — f(x) = G(h) = hg(h) always 
takes place? Yes, we can! By the Root-Factor Theorem, we may factor out h = h—0 
from the polynomial G(h) iff G(0) = 0. But G(0) = f(«+0)—f(x) = f(x)—f(x) = 
0. Thus some simple polynomial algebra ensures that we will be able to compute 
the derivative of any polynomial function, provided we assume the following fact. 
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Fact 4.2. For any polynomial function g, 
lim g(h) = g(a). 
Sa 


Differentiating polynomial functions directly from the definition is, evidently, some- 
what tedious. Perhaps we can establish some techniques to streamline the process? 
For instance, suppose we know the derivative of some function f: what can we say 
about the derivative of cf (a), where c is some real number? Let’s see: 


_ ef(@+h)—cf(x) _ ,. f(a+h) — f(x) 
pg Ao (Uf) 


If we assume lim, 44 cf(x) = climz-44 f(x), then we can complete this computation: 
the derivative of cf (x) is cf'(x). Let us again record our assumption about limits. 


Fact 4.3. If limysa f(@) = DL, then limg+4acf(x) = cL. 


This tells for instance that the derivative of 17x1° is 17(10x°) = 170x°. More gen- 
erally, this tells us that the derivative of the general monomial cx” is cnx”—!. 
Now what about sums? 


Let f and g be two differentiable functions. Then the derivative of f +g is 
fle +h) + 9(e+h) ~ fle)—9(2) _ 5, fle th)— Fle) , gle +h) —9(2) 


ae h +0 h h 


If we assume the limit of a sum is the sum of limits, we get 
(FAG Sf Pg 
Again, let’s record what we’ve used. 


Fact 4.4. If limg4a f(x) = L and lim,_,, g(x) = M, then lim,_,(f + g)(x) = 
L+M. 


EXERCISE 4.1. Show by induction that if fi,...,fn are functions with deriva- 
tives fi,...,f7, then (fit...t+ fn) =fit-..+ fi. 


Putting these facts together, we get an expression for the derivative of any polyno- 
mial function: if f(7) =ana"+...+a,x+ 49, then f’(x) = nayxz"~!+...+4,. In 
particular the derivative of a degree n polynomial is a polynomial of degree n — 1 
(and the derivative of a constant polynomial is the zero polynomial). 


3. Limits in Terms of Continuity 


We have been dancing around two fundamental issues in our provisional treatment 
of derivatives. The first is, of course, the notion of the limit of a function at a point. 
The second, just as important, is that of continuity at a point. 

In freshman calculus it is traditional to define continuity in terms of limits. A 
true fact which is not often mentioned is that this works just as well the other way 
around: treating the concept of a continuous function as known, one can define 
limits in terms of it. Since I think most people have at least some vague notion 
of what a continuous function is — very roughly it is that the graph y = f(x) isa 
nice, unbroken curve — and I[ know all too well that many students have zero in- 
tuition for limits, it seems to be of some value to define limits in terms of continuity. 


3. LIMITS IN TERMS OF CONTINUITY 57 


Let f : R > R be a function. For any « € R, f may or may not be continu- 
ous at x. We say f is simply continuous if it is continuous at x for every « € R.3u 


Here are some basic and useful properties of continuous functions. (Of course 
we cannot prove them until we give a formal definition of continuous functions.) 


Fact 4.5. a) Every constant function is continuous at every cE R. 
b) The identity function I(x) = x is continuous at every cE R. 
c) If f and g are continuous at x =c, then f +g and f-g are continuous at x =c. 
d) If f is continuous at x = c and f(c) £0, then ; is continuous at x =c. 
e) If f is continuous at « = c and g is continuous at « = f(c), then go f is 
continuous at x = Cc. 


From this relatively small number of facts many other facts follow. For instance, 
since polynomials are built up out of the identity function and the constant func- 
tions by repeated addition and multiplication, it follows that all polynomials are 
continuous at every c € R. Similarly, every rational function £ is continuous at 


every c in its domain, i.e., at all points c such that g(c) # 0. 


We now wish to define the limit of a function f at a point c € R. Here it is 
crucial to remark that c need not be in the domain of f. Rather what we need 
is that f is defined on some deleted interval J..5 about c: that is, there is some 
6 > 0 such that all points in (c — 6,c+ 6) except possibly at c, f is defined. To see 
that this a necessary business, consider the basic limit defining the derivative: 


/ _ f(@ +h) — fx) 
Fa) = fin EEO, 
Here «x is fixed and we are thinking of the difference quotient as a function of h. 
Note though that this function is not defined at h = 0: the denominator is equal 
to zero. In fact what we are trying to when differentiating is to find the most rea- 
sonable extension of the right hand side to a function which is defined at 0. This 
brings us to the following definition. 


Let f be a real-valued function defined on some subset D of R such that (c — 
6,c) U(c,e+ 4) is contained in D. Let L be a real number. Then lim,-_,, f(z) = L 
if the function f with domain (c — 6,c + 6) and defined as f(x) = f(a), x # ¢, 
f(c) = L, is continuous at c. 


Thus the limit LZ of a function as « — c is a value that if we “plug the hole” 
in the graph of the function y = f(x) by setting f(c) = L, we get a graph which is 
continuous — just think nicely behaved, for now — at « =c. 


Note that an immediate consequence of the definition is that if f(x) is itself con- 
tinuous at x = c, then it is already defined at c and lim,-,. f(z) = f(c). Thus the 
limit of a continuous function at a point is simply the value of the function at that 
point. This is very important! 


3The concept of continuity also makes sense for functions with domain a proper subset of R, 
but let’s save this for later. 
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In fact, we can now give a better account of what we have been doing when we 
compute f’(x). We start with the difference quotient fleth)- fe) which is defined 
for all sufficiently small h but not for h = 0. Then we manipulate / simplify the 
difference quotient until we recognize it as being equal, for all h 4 0, to some new 
function g(h) which is continuous at zero. (For instance, when f(x) = 2?, that 
new function was g(h) = 2% +h.) Then the limiting value is obtained simply by 
plugging in h = 0, ice., it is g(0). 


4. Continuity Done Right 


4.1. The formal defininition of continuity. 


Let D C R and f : D > R be a function. For c € R we say f is continuous 
at c if for all « > 0 there exists 6 > 0 such that (c—6,c+6) C D and for all x with 


le —e| <6, |f(0) — flo) <e. 
Morover we say that f is continuous if it is continuous at c for all c in its domain D. 


The bit about the domain D is traditionally left a bit more implicit, but since 
we are trying to be precise we may as well be completely precise. The condition 
is equivalent to requiring that there be some A > 0 such that f is defined on 
(c— A,c+A) and that for all « > 0, whenever we speak of 6 > 0 we always take it 
as implicit that 6 < A. 


In general, mathematical statements become more complex and harder to parse 
the more alternating quantifiers they have: i.e., statements of the form “There ex- 
ists x such that P(x)” or “For all 2, P(x)” have a simple logical structure (if P(x) 
is itself something reasonably simple, of course). Statements of the form “For all z, 
there exists y such that P(x, y)” and “There exists x such that for all y, P(x, y)” 
are a bit more complex; the untrained mind must stop to remind itself that they 
are not logically equivalent: e.g. if 2 and y are real numbers and P(x, y) is x > y 
then the first statement is true — for every real number, there exists a greatest 
real number — and the second statement is false — there is no real number which is 
greater than every eal number. The e-6 definition of continuity has three alternating 
quantifiers: for all, then there exists, then for all. In general, to fully comphrehend 
the meaning of statements this logically complex takes serious mental energy. 


Let us first talk about the geometric meaning of the statement: the inequality 
| f(a) — f(c)| < € means f(c) —€ < f(x) < f(c) +e: that is, it determines a hor- 
izontal strip centered at the horizontal line y = f(c) of width 2e. Similarly, the 
inequality |a—c| < 6 means c—6 < x < c+0, so determines a vertical strip centered 
at the vertical line x = c of width 26. Thus the statement is saying something about 
approximating both the y-values and the zx-values of the function. 


Now let us talk about the logical meaning of the statement and the sequence of 
quantifiers. We may think of it as a game: the first player chooses any € > 0 and 
thereby lays down a horizontal strip bounded above and below by the lines f(c) +€ 
and f(c) —e. The second player chooses a 6 > 0. Moreover, the second player wins 
if by restricting to x values lying between the vertical lines c— 6 and c+, the graph 
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of the function is trapped between the two vertical lines f(c)+€; otherwise the first 
player wins. Now the assertion that f is continuous at x = c is equivalent to the 
fact that the second player has a winning strategy: in other words, it is possible 
for her to win no matter which € > 0 the first player names. 


EXAMPLE 4.6. The constant function f:R—R by x f(x) =C is continu- 
ous. Forc € R ande > 0, we may choose 6 = 1 (or any positive number, in fact). 
Then 


jz —e| <5 = |f (a) - (| =|C-C] =0<e. 


EXAMPLE 4.7. The identity function I: R > R given by x + I(x) = « is 
continuous. Forc € R ande > 0, we may take 6 =e. Then 


jc—el <6 => |[I[(x)-I(o)| =|u-—el <b =e. 

EXAMPLE 4.8. We claim that every linear function is continuous. Namely, let 

m,b ER, and let L: RR be given by 
cr L(x) = ma +b. 
If m =0, then L is constant — cf. Example 4.6 — so we may assume m # 0. For 
cE€R ande>0 we may take 0 = TAy- Then 
€ 

|m| 

EXAMPLE 4.9. Let f: RR byx f(z) = 27. We claim that f is continu- 
ous. Letc € R and fixe > 0. If |x —c| <6 then 


j7—ce] <6 => |f@)—fO] = |ma+b—(me+b)| = |mlla—el < [md = |m|—— =. 


f(x) — f(Q)| = |x? - | = |x + ella — ¢| < |x + a6, 
so the task is to control |x + c| in terms of 6. Let us assume that 6 <1, i.e., 
ja—c| <1. 
By the Reverse Triangle Inequality we have 
Iz] — Jel] < |e—e] <1, 
50 
-1< |2|— |e] <1. 
Thus 
|a| < |e] +1. 


(One does not really need the Reverse Triangle Inequality here, as one can reason 
as follows: the distance of x from 0 is no larger than the sum of the distance of x 
from c and the distance of c from 0.) It follows that 


Jz +e] < |a] + |e] < (lel + 1) + |e] = Ie +1. 


€ 
6 = min (1, ; 
min ( an) 
Then 


joe] <5 => [f(e)-F(0)| < lets < @lel+1) min (1, 5-5) < @lel+t) (5-55) =< 


So we take 
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4.2. Basic properties of continuous functions. 


LEMMA 4.6. (Upper and Lower Bounds for Continuous Functions) 
Let f be continuous at x = c. 
a) For any € > 0, there exists 6 > 0 such that |x —c| < 6 implies |f(x)| < |f(o| +e. 
b) Suppose f(c) £0. Then for any a € (0,1), there exists 6 > 0 such that |jr—c| <6 
implies |f(2)| > alf(6)| 


PROOF. a) For any € > 0, there exists 6 > 0 such that |x — c| < 6 implies 
| f(x) — f(c)| < «. By the Reverse Triangle Inequality, 


If(x)| — IF()] S IF(®) - F()| <, 
so 
If(x)| < |f(c)| +e. 
b) We will prove the result for a = 5, leaving the general case as an extra credit 


problem. There exists 6 > 0 such that |x — c| < 6 implies |f(x) — f(o)| < LA) 
Again the Reverse Triangle Inequality implies 


OL LF@I< [Fe — FQ < YO 


9 ? 
or 


y(@)| > [4(o| — AO = HO 


THEOREM 4.7. Let f and g be functions andc ER. 
a) If f is continuous at c and AER, then Af is continuous at c. 
b) If f and g are both continuous at c then f +g is continuous at c. 
c) If f and g are both continuous at c then fg is continuous at c. 


d) If f and g are both continuous at c and g(c) £0, then . is continuous at c. 
e) If f is continuous at c and g is continuous at f(c) then go f is continuous at c. 


PROOF. For each part we work out the idea of the proof first and then translate 
it into a formal e-6 argument. 
a) Fix € > 0. We must show that there exists 6 > 0 such that |x — c| < 6 implies 
|Af(x) — Af(c)| < e. but |Af (x) — Af(c)| = |Al| f(x) — f(©)|. Moreover, precisely 
because f is continuous at c we may make the quantity | f(a) — f(c)— as small as 
we like by taking x sufficiently close to c. A quantity which we can make as small 
as we like times a constant can still be made as small as we like! 

Now formally: we may assume A ¥ 0 for othewise Af is the constantly zero 
function, which we have already proved is continuous. For any € > 0, since f is 
continuous at « = c there exists 6 > 0 such that |” —c| < 6 implies | f(a) — f(c)| < 


€ 


Tay" (Note what is being done here: by continuity, we can make | f(x) — f(c)| less 


than any positive number we choose. It is convenient for us to make it smaller than 
€ 


Tay? where € is a previously given positive number.) Then |z — c| < 6 implies 

c = 
i 
b) Fix e > 0. We must show that there exists 6 > 0 such that |x — c| < 6 implies 
|f(x) + g(x) — (F(e) + g(c))| < €. Now 


f(x) +9(2)—-(F(e)+9(©))| = IF (@)-FO)+(9(@)-9)| S$ |F@)-FOI+9@)-9)I- 


|Af(x) — Af(©)| = lAllf(@) — FOI < IAT 


€. 
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This is good: since f and g are both continuous at c, we can make | f(x) — f(c)| 
and |g(x) — g(c)| as small as we like by taking x sufficiently close to c. The sum of 
two quantities which can each be made as small as we like can be made as small as 
we like! 

Now formally: choose 6; > 0 such that |x — c| < 6; implies |f(a) — f(e)| < §. 
Choose 62 > 0 such that | — c| < d2 implies |g(x) — g(c)| < §. Let 6 = min(d;, 62). 
Then |x — c| < 6 implies |x — c| < 6) and |x — c| < 62, so 

€ 


Ifa) + g(a) - (F() + 9(0)I < IF@) ~ FO + lgl@) - gO < $+ 5 


c) Fix e > 0. We must show that there exists 6 > 0 such that |x — c| < 6 implies 
|f(x)g(x) — f(ce)g(c)| < €. The situation here is somewhat perplexing: clearly we 
need to use the continuity of f and g at c, and to do this it stands to reason that we 
should be estimating | f(x)9(x) — f(c)g(c)| in terms of | f(a) — f(c)| and |g(x) —g(c)|, 
but unfortunately we do not yet see these latter two expressions. So we force them 
to appear by adding and subtracting f(x)g(c): 


If(x)g(z) — Fg] = |f(@)9(@) — Fla)g(o) + Fea — Fg) 
< lf(@)llg@) — 9] + lg OllF(@) — FOL 


This is much better: |g(c)|| f(x) — f(c)| is a constant times something which can 
be made arbitrarily small, so can be made arbitarily small. Moreover in the term 
f(x) \|g(@) — g(c)| we can make |g(x) — g(c)| arbitarily small by taking « sufficiently 
close to c, and then, by continuity of f, |f(«x)| gets arbitrarily close to |f(c)|. So 
f(a)| is nonconstant but bounded, and something which is bounded times something 
which can be made arbitarily small can be made arbitrarily small! 
Now formally: 
Using Lemma 4.6a) and taking « = 1, there exists 6; > 0 such that |” —c| < 61 
implies |f(x)| < |f(c)| +1. There exists 62 > 0 such that |x —c| < 62 implies 
g(x) —g(c)| < May: Finally, there exists 63 > 0 such that |x —c| < d3 implies 
f(z) — fll < Teor (Here we are assuming that g(c) # 0. If g(c) = 0 then we 
simply don’t have the second term in our expression and the argument is similar 
but easier.) Taking 6 = min 41, d2, 63, for ja — c| < 6 then |x — c| is less than 0), 62 
and 63 so 
If(x)9(x) — F()g()l S IF(@)|lo(@) — 9) + lg(OIIF(@) — F(O)| 
€ € € € 
SO OO a) 2 
io 


d) Since ie f: a in light of part c) it will suffice to show that if g is continuous 


= €. 


at c and g(c) 4 0 then 4 is continuous at c. Fix e > 0. We must show that there 


exists 6 > 0 such that |x — c| < 6 implies 
| 1 1 I< 
ge) ge)“ 
Now 
ie Bj Eo) _ lal) Sao). 
gz) g(e) lglg Ig@)II9()| 


Since g is continuous at « = c, we can make the numerator |g(x) — g(c)| as small 
as we like by taking x sufficiently close to c. This will make the entire fraction as 
small as we like provided the denominator is not also getting arbitrarily small as x 
approaches c. But indeed, since g is continuous at c and g(c) 4 0, the denominator 
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is approaching |g(c)|?_ 4 0. Thus again we have a quantity which we can make 


arbitarily small times a bounded quantity, so it can be made arbitrarily small! 
Now formally: 
We apply Lemma 4.6b) with a = 4: there exists 6; > 0 such that |x —c| < 64 
implies |g(a)| > late) and thus also 

1 2 


@lgOl ~ 190 


Also there exists d2 > 0 such that |” — c| < d2 implies |g(x) — g(c)| < (gr) é. 
Take 6 = min(61, 62). Then |a — c| < 6 implies 


1 Tae! 1 ¥3 4 2 la(c)|? bee 
ae Cae). Gonot ae 101 < Trap ( 3 ) : 


e) Fix « > 0. Since g(y) is continuous at y = f(c), there exists y > 0 such that 
ly — f(c)| < 7 implies |g(y) — g(f(c))| < «. Moreover, since f is continuous at c, 
there exists 6 > 0 such that |a—c| < 6 implies |f (x) —f(c)| < y. Thus, if |a—c] < 6, 
If(x) — f(e)| = ly — f(o)| < 7 and hence 


la(f(x)) — 9( FO) = lay) — 9( FO) <€. 


COROLLARY 4.8. All rational functions are continuous. 


2° 


PROOF. Since rational functions are built out of constant functions and the 
identity by repeated addition, multiplication and division, this follows immediately 
from Theorem 4.7. 


Other elementary functions: unfortunately if we try to go beyond rational functions 
to other elementary functions familiar from precalculus, we run into the issue that 
we have not yet given complete, satisfactory definitions of these functions! For 
instance, take even the relatively innocuous f(x) = /z. We want this function to 
have domain [0, co), but this uses the special property of R that every non-negative 
number has a square root: we haven’t proved this yet! If a > Ois irrational we 
have not given any definition of the power function 7°. Similarly we do not yet 
have rigorous definitions of a® for a > 1, loga, sina and cos, so we are poorly 
placed to rigorously prove their continuity. However (following Spivak) in order 
so as not to drastically limit the supply of functions to appear in our examples 
and exercises, we will proceed for now on the assumption that all the above 
elementary functions are continuous. We hasten to make two remarks: 


REMARK 4.1. This assumption can be justified! That is, all the elementary 
functions above are indeed continuous at every point of their domain (with the small 
proviso that for power functions like \/x we will need to give a separate definition 
of continuity at an endpoint of an interval, coming up soon). And in fact we will 
prove this later in the course...much later. 


REMARK 4.2. We will not use the continuity of the elementary functions as an 
assumption in any of our main results (but only in results and examples explicitly 
involving elementary functions; e.g. we will use the assumed continuity of the sine 
function to differentiate it). Thus it will be clear that we are not arguing circularly 
when we finally prove the continuity of these functions. 
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5. Limits Done Right 
5.1. The Formal Definition of a Limit. 


In order to formally define limits, it is convenient to have the notion of a deleted 
interval [5 about a point c, namely a set of real numbers of the form 


0<|r—el<d 


for some 6 > 0. Thus J, consists of (c — 6,c) together with the points (c,c + 46), 
or more colloquially it contains all points “sufficiently close to c but not equal to c”. 


Now comes the definition. For real numbers c and L and a function f: DC R->R, 
we say lim,-,. f(x) = L if for every € > 0 there exists 6 > 0 such that for all x in 
the deleted interval I.,5 — i.e., for all x with 0 < |x —c| <6 — f is defined at x and 
|f(v) —L| <e. 


Among all the many problems of limits, perhaps the following is the most basic 
and important. 


THEOREM 4.9. The limit at a point is unique: that is, if L and M are two 
numbers such that lim,_,. f(x) = L and lim,,. f(z) = M, then L= M. 


Proor. Seeking a contradiction, we suppose L 4 M;; it is no loss of generality 
to suppose that L < M (otherwise switch L and M) and we do so. Now we take 
e= abt. in the definition of limit: since lim,_,. f(a) = L, there exists 6; > 0 such 
that 0 < |2 —c| < 6; implies | f(x) — L| < “S*; and similarly, since lim,_,, f(x) = 
M, there exists 6. > 0 such that 0 < |x —c| < 6: implies |f(«) — M| < 4. 
Taking 6 = min(0j, 62, then, as usual, for 0 < |” — c| < 6 we get both inequalities: 
M-L 

2 
M-L 


|f(x) —L| < 


|f(a) -— M| < 


However these inequalities are contradictory! Before we go further we urge the 
reader to draw a picture to see that the vertical strips defined by the two in- 
equalities above are disjoint: they have no points in common. Let us now check 
this formally: since | f(x) — L| < “54, f(z) < L+ %* = “. On the other 
hand, since | f(z) — M| < “54, f(x) > M — “44 = ““. Clearly there is not 
a single value of x such that f(x) is at the same time greater than and less than 
Me © ‘let alone a deleted interval around c of such values of x, so we have reached 
a contradiction. Therefore L = M. 


We have now given a formal definition of continuity at a point and also a formal 
definition of limits at a point. Previously though we argued that each of limits and 
continuity can be defined in terms of the other, so we are now in an “overdeter- 
mined” situation. We should therefore check the compatibility of our definitions. 


THEOREM 4.10. Let f: DCR-R, and letc ER. 
a) The function f is continuous at x = c iff it is defined at c and lim,-,. f(x) = f(c). 
b) We have lim,_,. f(x) = L iff f is defined in some deleted interval I,,5 around x 
and, defining f on (c—6,c+6) by f(x) = f(a), x #e, f(c) = L makes f continuous 
atxrT=c. 
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Proor. All the pedagogical advantage here comes from working through this 
yourself rather than reading my proof, so I leave it to you. 


5.2. Basic Properties of Limits. 


Most of the basic properties of continuity discussed above have analogues for limits. 
We state the facts in the following result. 


THEOREM 4.11. Let f and g be two functions defined in a deleted interval I.,5 
of a point c. We suppose that lim,,. f(x) = L and lim,-,. g(x) = M. 
a) For any constant A, lim,4- Af (a) = AL. 
b) We have limg. f(z) + 9(z) =L+M. 
c) We have lim,_,. f(x)g(z) = LM. 


d) If M #0, then lim, 4. 49 = #. 


We leave the proof to the reader. It is possible to prove any/all of these facts in 
one of two ways: (i) by rephrasing the definition of limit in terms of continuity and 


appealing to Theorem 4.7 above, or (ii) adapting the proofs of Theorem 4.7 to the 
current context. 


Now what about limits of composite functions? The natural analogue of Theo- 
rem 4.7e) above would be the following: 


If lim,_,. f(z) = L and lim,_,, g(x) = M, then lim,-_,. g(f(x)) = M. 
Unfortunately the above statement is not always true! 


Example 5.1: Let f(x) = 0 (constant function). Let g(x) be equal to 1 for x 4 0 
and g(0) = 0. Take c = 0. Then lim,-,o f(x) = 0 and lim,z_,9 g(x) = 1, but for all 
« ER, g(f(x)) = g(0) = 0, so lime +0 g(f(x)) = 0. 


This result can be repaired by requiring the continuity of the “outside” function 
g(x). Indeed the proof of the following result is almost identical to that of Theorem 
4.7e) above in which f is also continuous at c: in fact, it shows that the continuity 
of the “inside function” f was not really used. 


THEOREM 4.12. Let f,g be functions with lim,,. f(a) = L and g continuous 
at L. Then limz-+-9(f(x)) = g(L). 


PROOF. Fix € > 0. Since lim,-, ¢(<) g(y) = g(Z), there exists y > 0 such that 
ly — f(c)| < ¥ implies |g(y) — g(Z)| < e. Since lim,-_,. f(z) = L, there exists 
5 > 0 such that 0 < |x —c| < 6 implies |y — L| = |f(a) — L| < y and thus 
lg(F()) — 9(Z)| = |g(y) — 94) < 
One may rephrase Theorem 4.12 as: if g is continuous and lim,-,- f(x) exists, then 


lim (f(x) = g(tim f(2)). 


In other words, one can “pull a limit through a continuous function”. In this form 
the result is actually a standard one in freshman calculus. 


It happens that one can say exactly when the above statement about limits of 
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composite funtions holds. I don’t plan on mentioning this in class and you needn’t 
keep it in mind or even read it, but I recently learned that this question has a rather 
simple answer so I might as well record it here so I don’t forget it myself. 


THEOREM 4.13. (Marjanovié [MK09]) Suppose limz-4¢ f(x) = L and limz1 g(x) = 
M. The following are equivalent: 
(i) limz. 9(f(x)) = M. 
(ii) At least one of the following holds: 
a) g is continuous at L. 
b) There exists A > 0 such that for all0 < |u—c| <A, we have f(x) 4 L. 


PROOF. (i) ==> (ii): We will argue by contradiction: suppose that neither a) 
nor b) holds; we will show that limz-_.. g(f(x)) # M. Indeed, since b) dooes not 
hold, for every 6 > 0 there exists x with 0 < |x —c| <6 such that f(x) = L. For 
such x we have g(f(x)) = g(L). But since a) does not hold, g is not continuous at 
L, ie, M A g(L). Thus g(f(x)) = g(L) 4 M. Taking € = |g(L) — M| this shows 
that there is no 6 > 0 such that 0 < |x —c| < 6 implies |g(f(x)) — M| < «, so 
lime s< 9(f(2)) # M. 

(ii) == (i). The case in which a) holds — ie., g is continuous at L ~ is precisely 
Theorem 4.12. So it suffices to assume that b) holds: there exists some A > 0 such 
that 0 < |x —c| < A implies f(z) # L. Now fix € > 0; since lim,_,7, g(x) = M, 
there exists y > 0 such that 0 < |y—L| < y implies |g(y) — M| < «. Similarly (and 
familiarly), since lim,_,. f(x) = L, there exists 6; > 0 such that 0 < |x —¢| < 0) 
implies |f(«)—L| < y. Here is the point: for 0 < |z—c| < 41, we have | f(x)—L| < +. 
If in addition 0 < |f(x) — L| < y, then we may conclude that |g(f(z)) — M| < e. 
So our only concern is that pehaps f(x) = L for some c with 0 < |x —c¢| < 64, 
and this is exactly what the additional hypothesis b) allows us to rule out: if we 
take 6 = min(d1, A) then 0 < |a —c| < 6 implies 0 < |f(x) — L| < y and thus 
lg(f(2)) - M| <e. 


Remark 5.2: The nice expository article [MK09] gives some applications of the 
implication (ii)b) ==> (i) of Theorem 4.13 involving making an inverse change of 
variables to evaluate a limit. Perhaps we may revisit this point towards the end of 
the course when we talk about inverse functions. 


5.3. The Squeeze Theorem and the Switching Theorem. 


THEOREM 4.14. (Squeeze Theorem) Let m(x), f(x) and M(x) be defined on 
some deleted interval I° = (c— A,c+ A) — {c} about x =c. We suppose that: 
(i) For alla € I°, m(x) < f(x) < M(a), and 
(it) limz-.m(x) = lim,_,. M(x) = L. 
Then limz-+¢ f(x) = L. 


PrRoor. Fix « > 0. There exists 6; > 0 such that 0 < |a —c| < 6, implies 
|m(x) — L| < € and 52 > 0 such that 0 < |a — c| < 69 implies |M(x) — L| < €. Let 
56 = min(d), 62). Then 0 < |x — ¢| < 6 implies 

f(a) < M(x) <L+e 
and 

f(z) > m(z) >L-e, 
so L—e< f(x) < L +e, or equivalently | f(x) — L| <e. 
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EXAMPLE 4.10. For a > 0, define fa : [0,c0) > R by 
x*sin(t) 2x40 
fa(0) = 0. 


By our assumption about the continuity of sign and our results on continuity of 
rational functions and compositions of continuous functions, fa is continuous at 
allz £0. We claim that fa is continuous atx =0 iffa>0. 

Let a= 0. For x > 0 we have f(x) = sin(4). On any interval [M, oo), the 
sine function takes every value between —1 and 1, and thus on any interval (0,6) 
the function sin(+) takes every value between —1 and 1, so lim,_,9+ sin(+) does not 
exist. Thus fo is not continuous at 0. 

Leta >0. Since |siny| <1 for all real numbers y, for all xc 40 we have 


—2* < fa(z) <x. 


(16) fa(x) = 


Since ++ x is continuous on [0,00) we have 


lim «* = lim —x2* = 0, 
«wr 0o xL—-0o 


80 by the Squeeze Theorem we conclude 
lim fa(2) =0= fa(0). 
So fa is continuous at 0. 


EXAMPLE 4.11. Similarly, the function fy defined in (16) is differentiable at 
alla #0. We show that f., is differentiable at x = 0 iffa > 1. Since fa(0) = 0, 
fa is differentiable at 0 iff 


; fa(O+h) — fa(0) _ 4 fa(h) 
ay h pai 


exists (in which case it is f2(0)). We have 


ee. fas Zhe 1 
L= lim — =h*“sin(—). 
hoo h ns) 
Ifa <1 thena—1 <0, so limp_,9 h°~! = limp_so po = oo. Since sin(#) is 


bounded, the entire limit approaches infinity. If a > 1, then applying Example 4.10 
we get that the limit L exists iff a> 1. 


EXAMPLE 4.12. Let’s show that limz_,o aU tl 

Solution: This will be a “17th century solution” to the problem: i.e., we will as- 
sume that the trigonometric functions are continuous and use geometric reasoning, 
including properties of angles and arclength. 

Consider the unit circle and the point on it P, = (cosz,sinz). There is a right 
triangle T, with vertices (0,0), (cosz,0), P,. This right triangle is contained in 
the circular sector determined by all points on or inside the unit circle with angle 
between 0 and x. In turn this circular sector is contained in a second right triangle 
T2, with vertices (0,0), (1,0), (1,tanx). Now let us write down the inequalities ex- 
pressing the fact that since T, is contained in the circular sector which is contained 
in T, the area of T, is less than or equal to the area of the circular sector, which 
is less than or equal to the area of T>. 

The area of T; is (one half the base times the height) $ cosa sin x. The area of 
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the circular sector is s— times the area of the unit circle, or 5--m = 5. The area 
ee ly — lsinz : ; ‘ ie 
of Tz is 5tanz = 555. This gives us the inequalities 
; 1 sinz 
—cosxsinx < =x <= ; 
2 2 2 cos x 
or equivalently, for x #0, 
1 


< 


cosx < — < 
sin x cos x 


Taking reciprocals this inequality is equivalent to 


1 sin x 
< 
cos x x 


<cosz. 


Now we may apply the Squeeze Theorem: since cosx is continuous at 0 and takes 
the value 1 4 0 there, we have 


= lim cosx = 1. 
«z—0 COS ZX xz—0 


(17) lim ake 1. 
EXAMPLE 4.13. We will evaluate lim, +228 = 0. The idea is to use 


trigonometric identities to reduce this limit to an expression involving the limit 
(17). Here goes: 


. l-cosz . l—-cosx f/1+cosz . cos? x — 1 . —sin? x 
lim = lim = lim = lim 
e300 e300 1+cosz 30 a(1+cosxz) 2130 2(1+cosz) 
att ‘ —_ gj —0 
ay en sin x kim sing \ eae 
zr>0 2 z>01+cosx 2 
Of course we also have limy-+0 cose—} = —lim,_,9 2°82 = -—0=0. In summary: 
1 —cos sx —1 
(18) lim ——** = lim “** = 0. 
x—0 Cc x—0 x 


Before doing the next two examples we remind the reader of the composite angle 
formulas from trigonometry: for any real numbers 2, y, 


sin(a + y) = sinzcosy + cos asin y, 


cos(xz + y) = cosa cosy — sina siny. 
EXAMPLE 4.14. Jf f(x) =sinz, then we claim f’(x) = cosa. Indeed 


f(a@+h) — f(x) . sin(a +h) —sing . sinzcosh+coszsinh — sinx 
= lim = lim 


Tp one Ps 
ie) oe h h—0 h h0 h 
fst i 1—cosh ory ar sinh\ _ ss Geist aes ee 
=—sing | lim —>— cosa | lim —— ] = (—sin x) (cos x) - 1 = cosa. 
EXAMPLE 4.15. If f(x) =cosz, then we claim f'(x) = —sinx. Indeed 
pina f(a@+h)— f(x), cos(a+h)—cosa | 
se a h ner h 
i cos x cosh — sinx sinh — cosx 
i 


h-0 h 
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=— cose (jim a) —sin (sim ot) = (—cosz)-0+(—sinz)-1 = —sina. 
THEOREM 4.15. (Switching Theorem) Consider three functions f, 91,92 defined 

on some deleted interval I. of x =c. We suppose that: 

(4) lime g1(@) = limz-+¢ go(x) = L. 

(ii) For all x with 0 < |x —c¢| <A, either f(x) = gi (a) or f(x) = go(z). 

Then limg-ye f(x) = L. 


PRoor. Fixe > 0. Let 6; > 0 be such that 0 < |x—c| < 6) implies |gi(x)—L| < 
€, let d2 > 0 be such that 0 < |x —c| < de implies |go(x) — L| < e, and let 
56 = min(0), 62). Let x be such that 0 < |x — c| < 6. Then either f(x) = gi(z), in 
which case | f(a) — L| = |gi(x) — L| < €, or f(a) = go(x), in which case | f(x) — L| = 
|go(x) — L| < e. Either way, | f(x) — L| < é! 


EXAMPLE 4.16. Let f: R—R by 


_ je rEQ 
worm |", rER\Q 


We apply the Switching Theorem to show that f is continuous at 0. Indeed, put 
gi(“) = x and go(a) = —a. Then limzg-4o 91(x) = limz4o g2(x) = 0 and for all 
x, f(x) = gi(%) or f(x) = go(x). So by the Switching Theorem, f(0) = 0 = 


The previous example shows that a function may be very strangely behaved and 
still be continuous at a point. It is thus worth emphasizing that we are not really 
interested in functions which are continuous at certain points, but rather at func- 
tions which are continuous at every point of their domain. Such functions have 
many pleasant properties that we will prove later on in the course. 


5.4. Variations on the Limit Concept. 


Finally we consider three variations on the limit concept: one-sided limits, infi- 
nite limits, and limits at infinity. There is nothing really novel going on here, but 
we need to be sure that we can adapt our e-6 formalism to the variant notions of 
limit that apply in calculus. 


By way of introducing the first variant, consider the function f(x) = 7. We 
have said earlier that we are assuming for now that this function is continuous on 
its entire domain. But that statement glossed over a technicality which we now 
address. Namely, the domain of f is [0,co). So f cannot be continuous at 0 ac- 
cording to the definition that we gave because it is not defined on any open interval 
containing zero. Instead it is defined, for instance, on an interval of the form [0, 6) 
for 6 > 0: ie., this contains all points sufficiently close to 0 but greater than or 
equal to zero. 

A similar phenomenon arises when we consider the function f(x) = ,/z(1— 2), 
which has natural domain [0,1]. The function is defined at 1 but not in any open 
interval containing 1: only intervals of the form (1 — 6, 1]. 


This brings us to our definition. We say that a function f : D C R is right 
continuous at x = c if for every « > 0 there exists 6 > 0 such that f is defined on 
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[c,¢+ 6) and x € [c,c+6) => |f(x) — f(c)| < e. Similarly, we say that a function 
f : DCR is left continuous at x = c if for every € > 0 there exists 6 > 0 such 
that f is defined on (c — 6,c] and x € (c—4,c] = > |f(x) — f(d| <e. 


Finally we make the following definition: let J C R be any interval and let f: J R 
be a function. We say that a is a left endpoint of J if a € J and there is no x € I 
with x < a; similarly we say b is a right endpoint of J if b € J and there is 
no x € I with x > b. An interval J has at most one endpoint and at most one 
endpoint; all four possibilities of having / not having left / right endpoints are of 
course possible. Let us say c € J is an interior point if it is not a left endpoint or 
a right endpoint. Now we say that f : J >— R is continuous if: 


e f is continuous at c for each interior point c € I, 
e If J has a left endpoint a, then f is right continuous at a, and 
e If J had a right endpoint 6b, then f is left continuous at 6. 


Example: f(x) = /z is right continuous at x = 0. 


As above, it is necessary to require left/right continuity when discussing behav- 
ior at right/left endpoints of an interval. On the other hand one may still discuss 
left/right continuity at interior points of an interval, and it is sometimes helpful to 
do so. 


Example: Let f(x) = |x| be the greatest integer function. Then f is continu- 
ous at c for all c € R \ Z, whereas for any c € Z, f is right continuous but not left 
continuous at c. 


This example suggests the following simple result. 


PROPOSITION 4.16. For a function f: DC R—-R andce D, the following 
are equivalent: 
(i) f ts left continuous at c and right continuous at c. 
(it) f is continuous at c. 


We leave the proof to the reader. 
In a similar way we can define one-sided limits at a point c. 


We say lim, .- f(z) = LD — and read this as the limit as x approaches c from 
the left of f(a) is L — if for all « > 0 there exists 6 > 0 such that for all « with 
c<a<c+td, |f(z)-L| <e. 

We say lim, _..+ f(x) = LZ — and read this as the limit as x approaches c from 
the right of f(x) is L — if for all € > 0, there exists 6 > 0 such that for all x with 
c-b0<a<ce, |f(z)-Ll]<e. 


PROPOSITION 4.17. For a function f: DC R—-R andce D, the following 
are equivalent: 
(i) The left hand and right hand limits at c exist and are equal. 
(it) limz-+¢ f(x) exists. 
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Again we leave the proof to the reader. 


Example: Let f(x) = |x| be the greatest integer function, and let n € Z. Then 
lim, sn- f(x) =n—1 and lim,_,,+ f(z) =n, so f is not continuous at n. 


There is some terminology here — not essential, but sometimes useful. If for a 
function f and c € R, the left and right hand limits at c both exist but are unequal, 
we say that f has a jump discontinuity at c. If the left and right hand limts at 
c both exist and are equal — i-e., if lim,_,. f(z) = L exists — but still f(x) is not 
continuous c (this can happen if either f(c) # L or, more plausibly, if c is not in 
the domain of f) we say that f has a removable discontinuity at c. This ter- 
minology comes from our earlier observation that if we (re)define f at c to be the 
limiting value Z then f becomes continuous at c. One sometimes calls a discontinu- 
ity which is either removable or a jump discontinuity a simple discontinuity: i.e., 
this is the case whenever both one-sided limits exist at c but f is not continuous at c. 


Infinite limits: Consider lim,_,9 =. This limit does not exist: indeed, if it did, 
then there would be some deleted interval Jo,5 on which f is bounded, whereas 
just the opposite is happening: the closer x is to 0, the larger f(x) becomes. In 
freshman calculus we would say lim, x > Of(x) = co. And we still want to say 
that, but in order to know what we mean when we say this we want to give an €-d 
style definition of this. Here it is: 


We say lim,,- f(x) = oo if for all M € R, there exists 6 > 0 such that 0 < 
jc—cel<d = f(x) >M. 


Geometrically, this is similar to the ¢-d definition of limit, but instead of pick- 
ing two horizontal lines arbitrarily close to y = L, we pick one horizontal line which 
is arbitrarily large and require that on some small deleted interval the graph of 
y = f(x) always lie above that line. Similarly: 


We say lim,_,. f(a) = —oo if for all m € R, there exists 6 > 0 such that 0 < 
|x — c| < 6 implies f(x) < m. 


Example: Let us indeed prove that lim, _,9 = = oo. Fix M € R. We need to 
find 6 such that 0 < || < 6 implies 4 > M. It is no loss of generality to assume 


M > 0 (why?). Then 4 > M <> |2|< Tm so we may take 6 = wave 


CHAPTER 5 
Differentiation 


1. Differentiability Versus Continuity 
Recall that a function f : DC R-—- R is differentiable at a € D if 
jam £04) = fla) 
h-0 h 


exists, and when this limit exists it is called the derivative f’(a) of f at a. More- 
over, the tangent line to y = f(x) at f(a) exists if f is differentiable at a and is 
the unique line passing through the point (a, f(a)) with slope f’(a). 


Note that an equivalent definition of the derivative at a is 
sin £(2)— Fa) 
@a r—a 
One can see this by going to the e-6 definition of a limit and making the “substitu- 
tion” h = x —a: then 0 < |h| <6 = 0< |x-a| <0. 
THEOREM 5.1. Let f : DC R-R be a function, and leta € D. If f is 


differentiable at a, then f is continuous at a. 


PROOF. We have 


= (4,22 = £9) (ay 24) = H)-0=0 
Thus 
0 = lim(f(e) — f(a)) = (lim f(@)) = f(a), 


The converse of Theorem 5.1 is far from being true: a function f which is continu- 
ous at a need not be differentiable at a. An easy example is f(x) = |z| at a=0. 


In fact the situation is worse: a function f : R — R can be continuous every- 
where yet still fail to be differentiable at many points. One way of introducing 
points of non-differentiability while preserving continuity is to take the absolute 
value of a differentiable function. 
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THEOREM 5.2. Let f: DC R—-R be continuous ata € D. 
a) Then |f| is continuous at a. 
b) The following are equivalent: 
(i) f ts differentiable at a, and either f(a) #0 or f(a) = f’(a) =0. 
(it) |f| is differentiable at a. 


EXERCISE 5.1. Prove Theorem 8.31. 


2. Differentiation Rules 
2.1. Linearity of the Derivative. 


THEOREM 5.3. (Constant Rule) Let f be differentiable ata € R and C ER. 
Then the function Cf is also differentiable at a and 


(Cf)'(a) = Cf'(a). 
ProoF. There is nothing to it: 


h->0 h 


Het N10) _ opr 


h-0 h 


THEOREM 5.4. (Sum Rule) Let f and g be functions which are both differen- 
tiable ata € R. Then the sum f +g is also differentiable at a and 


(f +9)'(a) = f(a) + 9'(a). 


Proor. Again, no biggie: 


(F+9) (a) = Tm h = h i 
a f(at+h) — f(a) Loads glat+h)—gla) _ ! / 
= lim + lim - = f'(a)+g'(a) 


These results, simple as they are, have the following important consequence. 


COROLLARY 5.5. (Linearity of the Derivative) For any differentiable functions 
f and g and any constants C1, Co, we have 
(Cif + Cog)’ = Cif! + Cag’. 
2.2. Product Rule(s). 


THEOREM 5.6. (Product Rule) Let f and g be functions which are both differ- 
entiable ata € R. Then the product fg is also differentiable at a and 


(f9)'(a) = f'(a)g(a) + f(a)g'(a). 
PROOF. 


(Fa)'(a) = lim fla+h)g(a on) ~ f(a)g(a) 


— tom fat A)g(a +h) = f(a)g(a+ h) + (f@gla + h) = F(a)g(a)) 
h-0 h 


= (jim, LO M= FO) (a g(a +m) +0) (Jam 190), 
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Since g is differentiable at a, g is continuous at a and thus limp,9 g(a + h) = 
lim,_, ag(x) = g(a). The last expression above is therefore equal to 


f'(a)g(@) + f(@g'(a). 


Dimensional analysis and the product rule: Leibniz kept a diary in which he 
recorded his scientific discoveries, including his development of differential calculus. 
One day he got to the product rule and wrote a formula equivalent to 

(f9)' = fg’. 
But this entire entry of the diary is crossed out. Three days later there is a new 
entry with a correct statement of the product rule and the derivation, which Leibniz 
says he has known “for some time”. 

I confess that I have taken this story from a third-party account. Every once 
in a while I think about trying to verify it, but I am always stopped by the fact 
that if it turns out to be apocryphal, I don’t really want to know: it’s too good 
a story! It is fundamentally honest about the way mathematics is done and, es- 
pecially, the way new mathematics is created. That is, to go forward we make 
guesses. Often we later realize that our guesses were not only wrong but silly, to 
the extent that we try to hide them from others and perhaps even from ourselves. 
But the process of “silly guesswork” is vital to mathematics, and if the great ge- 
nius Gottfried von Leibniz was not above them, what chance do the rest of us have? 


It is natural, like Leibniz, to want to hide our silly guesses. However, they play an 
important role in teaching and learning. Many veteran instructors of higher math- 
ematics come to lament the fact that for the sake of efficiency and “coverage of the 
syllabus” we often give the student the correct answer within minutes or seconds of 
asking the question (or worse, do not present the question at all!) and thus deprive 
them of the opportunity to make a silly guess and learn from their mistake. What 
can we learn from “Leibniz’s” guess that (fg)’ = f’g’? Simply to write down the 
correct product rule is not enough: I want to try to persuade you that “Leibniz’s 
guess” was not only incorrect but truly silly: that in some sense the product rule 
could have turned out to be any number of things, but certainly not (fg)’ = f’9’. 


For this I want to follow an approach I learned in high school chemistry: di- 
mensional analysis. We can give physically meaningful dimensions (the more 
common, but less precise, term is “units”) to all our quantities, and then our for- 
mulas must manipulate these quantities in certain ways or they are meaningless. 


For a simple example, suppose you walk into an empty room and find on the 
blackboard a drawing of a cylinder with its radius labelled r and its height labelled 
h. Below it are some formulas, and this one jumps out at you: 27(rh + r?). You 
can see right away that something has gone wrong: since r and h are both lengths 
—say in meters — rh denotes a product of lengths — say, square meters. But r® is 
a product of three lengths — i.e., cubic meters. How on earth will we get anything 
physically meaningful by adding square meters to cubic meters? It is much more 
likely that the writer meant 27(rh + r), since that is a meaningful quantity of di- 
mension length squared (i.e., area). In fact, it happens to be the correct formula for 
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the surface area of the cylinder with the top and bottom faces included, although 
dimensional considerations won’t tell you that. 


Something similar can be applied to the “formula” (fg)’ = f’g’. Let f = f(t) 
and view t as being time, say measured in seconds. Since f’(t) is a limit of quo- 
tients fleth) = fo) its dimension is the dimension of f divided by time. Now suppose 
both f(t) and g(t) are lengths, say in meters. Then f’(t) and g’(t) both have units 
meters per second, so f’(t) - g’(t) has units meters squared per second squared. 
On the other hand, the units of (fg)’ are meters squared per second. Thus the 
“formula” (fg)’ = f’g’ is asserting that some number of meters squared per second 
is equal to some number of meters squared per second squared. That’s not only 
wrong, it’s a priori meaningless. 

By contrast the correct formula (fg)’ = f’g + fg’ makes good dimensional 
sense: as above, (fg)/ is meters squared per second; so is f’g and so is fg’, so we 
can add them to get a meaningful number of meters squared per second. Thus the 
formula at least makes sense, which is good because above we proved it to be correct. 


Taking these ideas more seriously suggests that we should look for a proof of the 
product rule which explicitly takes into account that both sides are rates of change 
of areas. This is indeed possible, but we omit it for now. 


Suppose we want to find the derivative of a function which is a product of not 
two but three functions whose derivatives we already know, e.g. f(x) = xsin ze”. 
We can — of course? — still use the product rule, in two steps: 
f'(x) = (asin ze*)’ = ((xsinz)e”)’ = (asin z)'e” + (asin z)(e”)’ 
= (a’ sinz + x(sinz)')e” +xsinze* = sinz + xcosze” + xsin ze”. 

Note that we didn’t use the fact that our three differentiable functions were z, 
sinx and e*® until the last step, so the same method shows that for any three 
functions f,, fo, fg which are all differentiable at x, the product f = fi fofs is also 
differentiable at a and 


f'(@) = fila) fol) fa(@) + fila) fa(@) fa(@) + fi(@) f(a) f3 (a). 
The following result rides this train of thought to its final destination. 
THEOREM 5.7. (Generalized Product Rule) Let n > 2 be an integer, and let 


fi,---;fn ben functions which are all differentiable ata. Then f = fi--- fn is also 
differentiable at a, and 


(19) (fas fn)(@) = fila) fala): fla) +--+ fila) +++ fr-1(@) fr, (@)- 
PROoF. By induction on n. 

Base Case (n = 2): This is precisely the “ordinary” Product Rule (Theorem 5.6). 

Induction Step: Let n > 2 be an integer, and suppose that the product of any n 

functions which are each differentiable at a € R is differentiable at a and that the 

derivative is given by (19). Now let fi,..., fn, fnti be functions, each differentiable 

at a. Then by the usual product rule 


(fi eal Fnfn+1) (a) = (ft vie fn) fn+1)’ (a) = (fi ole fn)’ (@) fn. (a) +1 (a) ees fn(@) fn41(@)- 


Using the induction hypothesis this last expression becomes 


(fi(@) fala) «++ fn(a) +... + fila) ++ fa-1(a) fr (@)) fro (a) + fi(@) + fn(@) Png (@) 
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= f(a) fo(a)--- fr(a)fn4i(a) +... + fila) +++ fr(a)fn41(@)- 
Example: We may use the Generalized Product Rule to give a less computationally 
intensive derivation of the power rule 

(2”)! = north 
for n € Z*. Indeed, taking f; = --- = f, = x, we have f(x) = 2” = fi --- fn, 80 
applying the Generalized Power rule we get 
(x) = (a)a---e+...4a@---2(a)’. 

Here in each term we have x’ = 1 multiplied by n — 1 factors of x, so each term 
evaluates to x”~!. Moreover we have n terms in all, so 


(2)! = nat. 


No need to dirty our hands with binomial coefficients! 


THEOREM 5.8. (Generalized Leibniz Rule) Letn € Z*, and let f,g be functions 
which are each n times differentiable ata € R. Then fg is is n times differentiable 


at a and 
n 


n = 
Gg => &@ pHgn-¥), 
k=0 
EXERCISE 5.2. Prove Theorem 5.8. 
THEOREM 5.9. (Quotient Rule) Let f and g be functions which are both differ- 


entiable ata € R, with g(a) #0. Then f is differentiable at a and 


@ (a) = $L"(@) = fa)s'(@) 
g ga)? , 

PROooF. Step 0: First observe that since g is continuous and g(a) # 0, there is 
some interval J = (a — 6,a +6) about a on which g is nonzero, and on this interval 


£ is defined. Thus it makes sense to consider the difference quotient 


flat h)/gla +h) — f(@)/9(@) 
h 


for h sufficiently close to zero. 
Step 1: We first establish the Reciprocal Rule, i.c., the special case of the Quo- 
tient Rule in which f(z) = 1 (constant function). Then 

1 


ty! 1 ere oe 
(2 =i = 


h-0 


~ tim SO — 9th) _ _ (;., Gath) —g(@)) (5. 1 _ =9'(a) 
ae hg(a)g(a + h) (im h ) (im g(a)g(a+ 5) g(a)? ~ 


We have again used the fact that if g is differentiable at a, g is continuous at a. 
Step 2: We now derive the full Quotient Rule by combining the Product Rule and 
the Reciprocal Rule. Indeed, we have 


(1) 7G Ve = Ha) + Flo) (Z) @ 
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LEMMA 5.10. Let f: DC R—-R. Suppose: 
(i) limz-4a f(x) exists, and 
(ti) There exists a number L € R such that for all 6 > 0, there exists at least one x 
with 0 < |x —a| <6 such that f(x) = L. 
Then limga f(x) = L. 


EXERCISE 5.3. Prove Lemma 5.10. (Suggestion: suppose limz-44 f(x) = MF 
L, and derive a contradiction by taking € to be small enough compared to |M — L|.) 


THEOREM 5.11. (Chain Rule) Let f and g be functions, and let a € R be such 
that f is differentiable at a and g is differentiable at f(a). Then the composite 
function go f is differentiable at a and 


(go f)'(a) =9'(F(@)) f(a). 
PROOF. Motivated by Leibniz notation, it is tempting to argue as follows: 


/ _ F(z) ~ 9 F(a) _ | (oF (2) — 9(F(@)) | ( fe) = F@ 
(go f)'(a) = lim 2 = tim ( )-( ) 
) L-a 


eer z—a za\ f(a) — fla 
(m2) om £0) 


Fa)>f(a) f(x) — f(a) wa Ea 
The replacement of “lim;4a... by limyye)+f(a)--.” in the first factor above is 
justified by the fact that f is continuous at a. However, this argument has a gap: 
when we multiply and divide by f(a) — f(a), how do we know that we are not 
dividing by zero?? The answer is that we cannot rule this out: it is possible for 
f(a) to take the value f(a) on arbitarily small deleted intervals around a: again, 
this is exactly what happens for the function f(x) of the above example near a = 0. 
I maintain that the above gap can be mended so as to give a complete proof. 
The above argument is valid unless for all 6 > 0, there is with 0 < |r—a| <6 
such that f(«) — f(a) = 0. In this case, it follows from Lemma 5.10 that if 


san £2) fla) 
=a T—a 

exists at all, it must be equal to 0. But we are assuming the above limit exists, since 
we are assuming f is differentiable at a. So f’(a) = 0, and therefore, since the Chain 
Rule reads (g 0 f)/(a) = g'(f(a)) f(a), here we are trying to show (go f)/(a) = 0. 
For z € R we have two possibilities: the first is f(z) — f(a) = 0, in which case 
also g(f(x)) — 9(f(a)) = o(f(a)) — g(f(a)) = 0, so the difference quotient is zero at 
these points. The second is f(x) — f(a) 4 0, in which case 


g( f(x) = 9(F(@)) , fle) = F(@ 

a(F(@)) —9(f(@) =e 

holds, and the above argument shows this expression tends to g’(f(a))f’(a) = 0 
g(f(@))~9( F(a) 
equal to!) zero. Thus the limit tends to zero no matter which alternative obtains. 
Somewhat more formally, if we fix « > 0, then the first step of the argument 
shows that there is 6 > 0 such that for all x with 0 < |x —a| < 6 such that 
f(x) — f(a) £0, | aetedstF(@)) | <e. On the other hand, when f(x) — f(a) = 0, 


as x + a. So whichever holds, the difference quotient is close to (or 
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then | stile s(Fl@)) | = 0, so it is certainly less than e! Therefore, all in all we have 


0<|r—-al<6 = | siflen—s(Fl@)) | <€, so that 


fam $d (2) = 9F(@)) 


@—a x-a 


=0=9'(f(a)) f(a). 


3. Optimization 


3.1. Intervals and interior points. 


At this point I wish to digress to formally define the notion of an interval on 
the real line and and interior point of the interval. ... 


3.2. Functions increasing or decreasing at a point. 


Let f : D> R be a function, and let a be an interior point of D. We say that f is 
increasing at a if for all x sufficiently close to a and to the left of a, f(x) < f(a) 
and for all sufficiently close to a and to the right of a, f(x) > f(a). More formally 
phrased, we require the existence of a 6 > 0 such that: 

e for all z with a—d <a <a, f(x) < f(a), and 

e for all x witha <a<a+t6, f(x) > f(a). 


We say f is decreasing at a if there exists 6 > 0 such that: 
e for all z witha—d<a<a, f(x) > f(a), and 
e for all x witha <a<a+t6, f(x) < f(a). 


We say f is weakly increasing at a if there exists 6 > 0 such that: 
e for all z witha—d<a<a, f(x) < f(a), and 
e for all x witha <a<a+t6, f(x) > f(a). 


EXERCISE 5.4. Give the definition of “f is decreasing at a”. 


EXERCISE 5.5. Let f : I > R, and let a be an interior point of I. 


a) Show that f is increasing at a iff —f is decreasing at a. 
b) Show that f is weakly increasing at a iff —f is weakly decreasing at a. 


EXAMPLE 5.1. Let f(x) = ma +b be a linear function. Then for anya eR: f 
is increasing at a iffm > 0, f is weakly increasing at a iffm > 0, f is decreasing 
at a iff m <0, and f is weakly decreasing at a iffm <0. 


EXAMPLE 5.2. Let n be a positive integer, and let f(x) =a". Then: 
e If x is odd, then for alla ER, f(x) is increasing at a. 
e If x is even, then ifa <0, f(x) is decreasing at a, ifa > 0 then f(a) ts increasing 
at a. When n is even f is neither increasing at 0 nor decreasing at 0 because for 
every nonzero x, f(x) > 0 = f(0).! 


If one looks back at the previous examples and keeps in mind that we are supposed 
to be studying derivatives (!), one is swiftly led to the following fact. 


lWe do not stop to prove these assertions as it would be inefficient to do so: soon enough 
we will develop the right tools to prove stronger assertions. But when given a new definition, it is 
always good to find one’s feet by considering some examples and nonexamples of that definition. 
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THEOREM 5.12. Let f: 1 +R. Suppose f is differentiable ata € I°. 
a) If f(a) > 0, then f is increasing at a. 
b) If f'(a) < 0, then f is decreasing at a. 
c) If f'(a) = 0, then no conclusion can be drawn: f may be increasing at a, de- 
creasing at a, or neither. 


PROOF. a) We use the e-6 interpretation of differentiability at a to our ad- 
vantage. Namely, take « = f’(a): there exists 6 > 0 such that for all x with 
0<|r—al <4, | f@-F@) — f'(a)| < f'(a), or equivalently 
f(x) = f(a) 


< 2f'(a). 


In particular, for all x with 0 < |z—al < 4, La) Sta) > 0, so: ifa >a, f(x)—f(a) > 
0, ie., f(x) > f(a); and if « <a, f(x) — f(a) <O,ie., f(x) < f(a). 

b) This is similar enough to part a) to be best left to the reader as an exercise. 
c) If f(z) = x, then f’(0) = 0 but f is increasing at 0. If f(x) = —a, then 
f'(0) = 0 but f is decreasing at 0. If f(z) = x7, then f’(0) = 0 but f is neither 
increasing nor decreasing at 0. 


2 


3.3. Extreme Values. 


Let f: D—R. We say M € R is the maximum value of f on D if 
(MV1) There exists  € D such that f(x) = M, and 
(MV2) For all « € D, f(x) < M. 


It is clear that a function can have at most one maximum value: if it had more 
than one, one of the two would be larger than the other! However a function need 
not have any maximum value: for instance f : (0,00) > R by f(x) = 4 has no 
maximum value: lim, _,9+ f(a) = co. 


Similarly, we say m € R is the minimum value of f on D if 
(mV1) There exists « € D such that f(x) =m, and 
(mV2) For all « € D, f(x) > m. 


Also, a function can have at most one minimum value but need not have any. The 
function f : R\{0} > R by f(x) = 4 has no minimum value: lim,_,9- f(z) = —0o. 


EXERCISE 5.6. For a function f : D—R, the following are equivalent: 
(i) The f assumes a maximum value M, a minimum value m, and we have 
M=m. 
(ii) The function f is constant. 


Recall that f : D — R is bounded above if there is B € R such that for all x € D, 
f(x) < B. A function is bounded below if there is b € R such that for all x € D, 
f(x) > b. A function is bounded if it is both bounded above and bounded below: 
equivalently, there exists B > 0 such that for all « € D, |f(a)| < B: ie., the graph 
of f is “trapped between” the horizontal lines y = B and y = —B. 


2Two ways to go: (i) Revisit the above proof flipping inequalities as appropriate. (ii) Use the 
fact that f is decreasing at a iff —f is increasing at a and f’(a) <0 <=> (—f)/(a) > 0 to apply 
the result of part a). 
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EXERCISE 5.7. Let f : D— R be a function. 


a) Show: if f has a maximum value, it is bounded above. 
b) Show: if f has a minimum value, it is bounded below. 


EXERCISE 5.8. a) If a function has both a maximum and minimum value 
on D, then it is bounded on D: indeed, if M is the maximum value of f 
and m is the minimum value, then for all x € B, |f(x)| < max(|m|, |M]). 

b) Give an example of a bounded function f : R > R which has neither a 
maximum nor a minimum value. 


We say f assumes its maximum value at a if f(a) is the maximum value of f 
on D, or in other words, for all « € D, f(x) < f(a). Simlarly, we say f assumes 
its minimum value at a if f(a) is the minimum value of f on D, or in other 
words, for all x € D, f(x) > f(a). 

EXAMPLE 5.3. The function f(x) = sina assumes its maximum value atx = 5, 
because sin} =1, and 1 is the maximum value of the sine function. Note however 
that 5 is not the only x-value at which f assumes its maximum value: indeed, the 
sine function is periodic and takes value 1 precisely at x = 5 + 2nn for n € Z. 
Thus there may be more than one x-value at which a function attains its maximum 
value. Similarly f attains its minimum value at 2 = 3% Fes) =-1 and f takes 


2 2 
no smaller values — and also at x = oe +2nn forn € Z. 


EXAMPLE 5.4. Let f : R > R by f(z) = 2° +5. Then f does not assume 
a mazimum or minimum value. Indeed, limg—o0. f(%) = co and limg4—. f(r) = 
—oo. 


EXAMPLE 5.5. Let f : [0,2] + R be defined as follows: f(z) =x+1,0<a<1. 
f(z) =1, c¢=1. 
f(@)=a2-1,l<a<2. 
Then f is defined on a closed, bounded interval and is bounded above (by 2) and 
bounded below (by 0) but does not have a maximum or minimum value. Of course 
this example of a function defined on a closed bounded interval without a maximum 
or minimum value feels rather contrived: in particular it is not continuous at x = 1. 


This brings us to one of the most important theorems in the course. 


THEOREM 5.13 (Extreme Value Theorem). Let f : [a,b] + R be continuous. 
Then f has a maximum and minimum value, hence is bounded above and below. 


The proof will be given in the next chapter, following a discussion of completeness. 


Ubiquitously in (pure and applied) mathematics we wish to optimize functions: 
that is, find their maximum and or minimum values on a certain domain. Unfor- 
tunately, as we have seen above, a general function f : D > R need not have a 
maximum or minimum value! But the Extreme Value Theorem gives rather mild 
hypotheses on which these values are guaranteed to exist, and in fact is a useful 
tool for establishing the existence of maximia / minima in other situations as well. 


EXAMPLE 5.6. Let f : R > R be defined by f(x) = x?(ax—1)(a—2). Note that f 
does not have a maximum value: indeed limy-+00 f() = limz-+-co = ©. However, 
we claim that f does have a minimum value. We argue for this as follows: given 
that f tends to oo with |x|, there must exist A > 0 such that for all x with |x| > A, 
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f(x) > 1. On the other hand, if we restrict f to [-A, A] we have a continuous 
function on a closed bounded interval, so by the Extreme Value Theorem it must 
have a minimum value, say m. In fact since f(0) = 0, we see that m < 0, so in 
particular m <1. This means that the minimum value m for f on [—A, A] must 
in fact be the minimum value for f on all of R, since at the other values — namely, 
on (—o0, —A) and (A, oo), f(x) >1>0>m. 

We can be at least a little more explicit: a sign analysis of f shows that f is 
positive on (—00, 1) and (2,00) and negative on (1,2), so the minimum value of f 
will be its minimum value on [1,2], which will be strictly negative. But exactly what 
is this minimum value m, and for which x value(s) does it occur? Stay tuned: we 
are about to develop tools to answer this question! 


3.4. Local Extrema and a Procedure for Optimization. 


We now describe a type of “local behavior near a” of a very different sort from 
being increasing or decreasing at a. 


Let f : D > R be a function, and let a € D. We say that f has a local 
maximum at a if the value of f at a is greater than or equal to its values at 
all sufficiently close points x. More formally: there exists 6 > 0 such that for all 
xcéeD,|xc-al<d = f(x) < f(a). Similarly, we say that f has a local 
minimum at a if the vaalue of f at a is greater than or equal to its values at 
all sufficiently close points 2. More formally: there exists 6 > 0 such that for all 
ceED,\r—al<6 = f(x) > f(a). 


THEOREM 5.14. Let f : D CR, and let a be an interior point of a. If f is 
differentiable at a and has a local extremum — 1.e., either a local minimum or a 
local maximum — at x =a, then f'(a) =0. 


ProoF. Indeed, if f’(a) 4 0 then either f’(a) > 0 or f’(a) < 0. 
If f’(a) > 0, then by Theorem 5.12, f is increasing at a. Thus for x slightly smaller 
than a, f(x) < f(a), and for « slightly larger than a, f(x) > f(a). So f does not 
have a local extremum at a. 
Similarly, if f’(a) < 0, then by Theorem 5.12, f is decreasing at a. Thus for x 
slightly smaller than a, f(x) > f(a), and for « slightly larger than a, f(x) < f(a). 
So f does not have a local extremum at a. 


THEOREM 5.15. (Optimization Theorem) Let f : I > R be continuous. Suppose 
that f attains a minimum or maximum value at x =a. Then a is either: 
(i) an endpoint of I, 
(ii) a stationary point: f'(a) =0, or 
(itt) a point of nondifferentiability. 


ProoF. This is an immediate consequence of Theorem 5.14. 


EXERCISE 5.9. State and prove a version of Theorem 5.15 for the maximum 
value. 


Often one lumps cases (ii) and (iii) of Theorem 5.15 together under the term critical 
point (but there is nothing very deep going on here: it’s just terminology). Clearly 
there are always exactly two endpoints. In favorable circustances there will be only 
finitely many critical points, and in very favorable circumstances they can be found 
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exactly: suppose they are ci,...,Cn. (There may not be any critical points, but 
that only makes things easier...) Suppose further that we can explicitly compute 
all the values f(a), f(b), f(c1),---, f(¢n). Then we win: the largest of these values 
is the maximum value, and the smallest is the minimum value. 


EXAMPLE 5.7. Let f(x) = x?(a—1)(a — 2) = x* — 3x3 + 2x. Above we argued 
that there is a A such that |x| > A => |f(x)| > 1: let’s find such a A explicitly. 
We intend nothing fancy here: 

f(x) =a* — 3a? + 22? > o* — 323 = 23 (2 — 3). 
So if x > 4, then 
a(x —3)>4°-1=64>1. 
On the other hand, if x < —1, then x <0, so —3x? > 0 and thus 
f(z) > 244 20? = 27 (2? +2) >1-3=3. 
Thus we may take A = 4. 


Now let us try the procedure of Theorem 5.15 out by finding the maximum and 
minimum values of f(x) = «4 — 323 + 22? on [—4, 4]. 


Since f is differentiable everywhere on (—4,4), the only critical points will be the 
stationary points, where f’(7) = 0. So we compute the derivative: 
f'(z) = 4a? — 9a? + 4x = 2 (4a? — 9a + 4). 


9+V1 
8 


NI 


The roots are x = 


, or, approximately, 

x, + 0.6094..., v2 = 1.604.... 
f(a1) =0.2017..., f(v2) = —0.619.... 

Also we always test the endpoints: 

f(—4) = 480, f(4) = 96. 


So the maximum value is 480 and the minimum value is —.619.... 


4. The Mean Value Theorem 
4.1. Statement of the Mean Value Theorem. 


Our goal in this section is to prove the following important result. 


THEOREM 5.16. (Mean Value Theorem) Let f : [a,b] > R be continuous on 

[a,b] and differentiable on (a,b). Then there is at least one c witha <c<b and 
f'(c) _ f(0) — f(a) 
b-—a 

Remark:? I still remember the calculus test I took in high school in which I was 
asked to state the Mean Value Theorem. It was a multiple choice question, and I 
didn’t see the choice I wanted, which was as above except with the subtly stronger 
assumption that fp(a) and fi; (b) exist: i.e., f is one-sided differentiable at both 
endpoints. So I went up to the teacher’s desk to ask about this. He thought for 
a moment and said, “Okay, you can add that as an answer if you want”, and so 
as not to give special treatment to any one student, he announced to all that he 


3Please excuse this personal anecdote. 
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was adding a possible answer to the Mean Value Theorem question. So I marked 
my added answer, did the rest of the exam, and then had time to come back to 
this question. After more thought I decided that one-sided differentiability at the 
endpoints was not required. So in the end I selected this pre-existing choice and 
submitted my exam. As you can see, my final answer was correct. But many other 
students figured that if I had successfully lobbied for an additional answer then 
“my” answer was probably correct, so they changed their answer from the correct 
answer to my added answer. They were not so thrilled with me or the teacher, but 
in my opinion he behaved admirably: talk about a “teachable moment”! 


One should certainly draw a picture to go with the Mean Value Theorem, as it 
has a very simple geometric interpretation: under the hypotheses of the theorem, 
there exists at least one interior point c of the interval such that the tangent line 
at c is parallel to the secant line joining the endpoints of the interval. 

And one should also interpret it physically: if y = f(x) gives the position of 
a particle at a given time zx, then the expression £() fla) is nothing less than the 
average velocity between time a and time b, whereas the derivative f’(c) is the in- 
stantaneous velocity at time c, so that the Mean Value Theorem says that there is at 


least one instant at which the instantaneous velocity is equal to the average velocity. 


Example: Suppose that cameras are set up at checkpoints along an interstate 
highway in Georgia. One day you receive timestamped photos of yourself at two 
checkpoints. The two checkpoints are 90 miles apart and the second photo is taken 
73 minutes after the first photo. You are issued a ticket for violating the speeed 
limit of 70 miles per hour. Your average velocity was (90 miles) / (73 minutes) 
- (60 minutes) / (hour) + 73.94 miles per hour. Thus, although no one saw you 
violating the speed limit, they may mathematically deduce that at some point your 
instantaneous velocity was over 70 mph. Guilt by the Mean Value Theorem! 


4.2. Proof of the Mean Value Theorem. 


We will deduce the Mean Value Theorem from the (as yet unproven) Extreme 
Value Theorem. However, it is convenient to first establish a special case. 


THEOREM 5.17. (Rolle’s Theorem) Let f : [a,b] + R. We suppose: 
(i) f ts continuous on [a, b]. 
(ii) f is differentiable on (a,b). 
(wii) f(a) = f(b). 


Then there exists c witha <c<_b and f’(c) =0. 


PRooF. By Theorem 5.13, f has a maximum M and a minimum m. 
Case 1: Suppose M > f(a) = f(b). Then the maximum value does not occur 
at either endpoint. Since f is differentiable on (a, 6), it must therefore occur at a 
stationary point: i-e., there exists c € (a,b) with f’(c) =0. 

Case 2: Suppose m < f(a) = f(b). Then the minimum value does not occur at 
either endpoint. Since f is differentiable on (a,b), it must therefore occur at a 
stationary point: there exists c € (a,b) with f’(c) = 0. 

Case 3: The remaining case is m = f(a) = M. Then f is constant and f’(c) = 0 
at every point c € (a,b). 
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To deduce the Mean Value Theorem from Rolle’s Theorem, it is tempting to tilt 
our head until the secant line from (a, f(a)) to (b, f(b)) becomes horizontal and 
then apply Rolle’s Theorem. The possible flaw here is that if we start a subset in 
the plane which is the graph of a function and rotate it too much, it may no longer 
be the graph of a function, so Rolle’s Theorem does not apply. 

The above objection is just a technicality. In fact, it suggests that more is true: 
there should be some version of the Mean Value Theorem which applies to curves 
in the plane which are not necessarily graphs of functions. Nevertheless formalizing 
this argument is more of a digression than we want to make, so the “official” proof 
that follows is (slightly) different. 


Proof of the Mean Value Theorem: Let f : [a,b] + R be continuous on [a,b] and 
differentiable on (a,b). There is a unique linear function L(a) such that L(a) = f(a) 
and L(b) = f(b): indeed, L is nothing else than the secant line to f between (a, f(a)) 
and (b, f(b)). Here’s the trick: by subtracting L(a) from f(x) we reduce ourselves 
to a situation where we may apply Rolle’s Theorem, and then the conclusion that 
we get is easily seen to be the one we want about f. Here goes: define 


g(x) = f(x) — L(z). 
Then g is defined and continuous on [a,}], differentiable on (a,b), and g(a) = 
f(a) —L(a) = f(a)— f(a) =0= f(6)— f(®) = fF) — LQ) = g(0). Applying Rolle’s 
Theorem to g, there exists c € (a,b) such that g’(c) = 0. On the other hand, since 
f(b)—f (a) 
b-a 


L is a linear function with slope , we compute 


0=g'(c) =f -L(Q =f" - ee 
and thus 


f= 


4.3. The Cauchy Mean Value Theorem. 


We present here a generalization of the Mean Value Theorem due to A.L. Cauchy. 
Although perhaps not as fundamental and physically appealing as the Mean Value 
Theorem, it has its place: later we will use it to prove L’H6pital’s Rule. 


THEOREM 5.18 (Cauchy Mean Value Theorem). Let f,g : [a,b] > R be con- 
tinuous and differentiable on (a,b). Then there exists c € (a,b) such that 
(20) (f(0) — Fla))9'(©) = (g(b) — g(a) FCC). 


PROOF. Case 1: Suppose g(a) = g(b). By Rolle’s Theorem, there is c € (a,b) 
such that g'(c) = 0. For this c, both sides of (20) are zero, hence they are equal. 
Case 2: Suppose g(a) 4 g(b), and define 

fO)-f 2) 
h(x) = f(x) - (ae g(x). 
(x) = f(@) = Ge) (x) 
Then h is continuous on |a, b], differentiable on (a,b), and 
V2 


oe f(a)(g(b) — g(a)) — g(a) (f(b) — f(a) 
g(b) — g(a) g(b)—gla) 
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n(b) = LOM) = a@) = 9)FO) = F@) _ flea) = OF) 
g(b) — g(a) g(b)— g(a)? 
so h(a) = h(b).* By Rolle’s Theorem there exists c € (a,b) with 


0 =n) =s(0) - (AP F2) (0, 


or equivalently, 


EXERCISE 5.10. Which choice of g recovers the “ordinary” Mean Value Theo- 
rem? 


The Cauchy Mean Value Theorem can be used to vindicate the “head-tilting argu- 
ment” contemplated before the formal proof of the Mean Value Theorem. To do 
so takes us a bit outside our wheelhouse (i.e., a seriously theoretical approach to 
single variable calculus), but it is interesting enough to be worth sketching here. 

First recall (if possible) the notion of a parameterized curve: this is given 
by a function f with domain an interval J in R and codomain the plane R?: we 
may write v : t +> (x(t), y(t)), where t + a(t) and y +> y(t) are each functions 
from I to R. We say that v is continuous at t if both x and y are continuous at t. 
Further, when both z and y are differentiable at t we decree that v is differentiable 
at t and set v’(t) = (a(t), y/(t)), which we think of as being a vector in the plane 
with tail at the point (a(t), y(t)). When a’(t) = y(t) = 0 we get the zero vector, 
which we don’t want: let’s call this a singular point. For any nonsingular point 
(a(t), y(t)) we can define the tangent line to be the unique line passing through 
the points (x(t), y(t)) and (a(t) + a'(t), y(t) + y’(t)). (When 2’(t) 4 0, we can 
define the tangent line in a more familiar way, as the unique line passing through 
(a(t), y(t)) with slope ut. the above definition is phrased so as to make sense also 
if x’(t) = 0 and y/(t) #0, in which case we get a vertical tangent line.) 


EXAMPLE 5.8. Let v : [0,27] + R? by v(t) = (cost, sint). This point traces 
out a path which winds once around the unit circle counterclockwise. We have 
v'(t) = (—sint,cost): since for all t, (—sint)? + (cost)? = 1, in particular v'(t) 
is never zero: there are no singular points. At the points t = 0,7,27, x'(t) = 0 
and the tangent line is vertical at the points (£1,0) (as you would certainly expect 
from contemplation of the unit circle). When t = 0, 5,7, an 2n, one of (a(t), y(t)) 
and (x'(t), y’(t)) is horizontal and the other is vertical, so in particular the tangent 
line at v(t) is perpendicular to u(t). This holds at all other points as well, e.g. by 


noting that the slope of the radial line from (0,0) to v(t)) is ue = 24 and the 


y(t) _ —cost 
a(t) ~ sint 
are opposite reciprocals. Thus we get a derivation using calculus of a familiar (or 


slope of the tangent line to v(t) is and observing that these two slopes 


so I hope!) fact from elementary geometry. 


Here is a version of the Mean Value Theorem for nonsingular parameterized curves. 


4Don’t be so impressed: we wanted a constant C such that if h(x) = f(«) — Cg(x), then 
h(a) = h(b), so we set f(a) — Cg(a) = f(b) — Cg(b) and solved for C. 
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THEOREM 5.19 (Parametric Mean Value Theorem). Let v : I > R?, tH 
(a(t), y(t)), be a nonsingular parametrized curve in the plane, i.e., for all t € I, 
x'(t) and y'(t) both exist and are not both 0. Leta <b€I, and suppose that either 
x(a) # x(b) or y(a) # y(b). Then there is c € (a,b) such that the tangent vector 
(a2'(c), y/(c)) at c is parallel to the secant line from (a, f(a)) to (b, f(b)). 


PROOF. We apply the Cauchy Mean Value Theorem with f = x and g = y: 
there is c € (a,b) such that 
(2(b) — x(a))y'(c) = (y(0) — y(a))a"(e). 
By assumption, 2’(c) and y’(c) are not both 0. 
Case 1: Suppose 2’(c) # 0. Then we must have x(b) — x(a) ¥ 0: if not, 
0 = (a(b) — x(a))y"(e) = (y(b) — y(a))2"(e) 
and thus, since x’(c) # 0, y(b) — y(a) # 0, and then we have x(a) = x(b) and 
y(a) = y(b), contrary to our hypothesis. So we may divide to get 
y'(©) _ y(b) = y@) 
a(e) — x(b) — x(a)’ 
which says that the tangent line to v at c has the same slope as the secant line 
between (x(a), y(a)) and (a(b), y(b)), hence these two lines are parallel. 
Case 2: Suppose z/(c) = 0, ie., the tangent line to v at c is vertical. By our 
nonsingularity assumption y'(c) 4 0, so 
0 = (y(b) — y(a))2"(c) = (w() — 2(a))y'(e) 
(y(a), y(b)) is vertical, 


~S 
a 
° 


implies x(a) = x(b), so the secant line from (x(a), x(b) 
hence the two lines are parallel. 


Conversely, it is possible (indeed, similar) to deduce the Cauchy Mean Value The- 
orem from the Parametric Mean Value Theorem: try it. Thus really we have one 
theorem with two moderately different phrasings, and indeed it is common to also 
refer to Theorem 5.19 as the Cauchy Mean Value Theorem. 


5. Monotone Functions 
A function f : J > R is monotone if it is weakly increasing or weakly decreasing. 


5.1. The Monotone Function Theorems. 


The Mean Value Theorem has several important consequences. Foremost of all 
it will be used in the proof of the Fundamental Theorem of Calculus, but that’s for 
later. At the moment we can use it to establish a criterion for a function f to be 
monotone on an interval in terms of sign condition on f’. 


THEOREM 5.20 (First Monotone Function Theorem). Let I be an open interval, 
and let f: I +R be a function which is differentiable on I. 

a) Suppose f'(x) > 0 for alla € I. Then f is increasing on I: for all 
1,02 € I with x1 < x2, f(x1) < f(x). 

b) Suppose f'(a) >0 for alla EI. Then f is weakly increasing on I: for all 
1,02 € I with x1 < x2, f(a.) < f(x). 

c) Suppose f'(x) < 0 for alla € I. Then f is decreasing on I: for all 
L1,%q inl with x, < x2, f(x1) > f(x2). 
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d) Suppose f’() <0 for alla € I. Then f is weakly decreasing on I: for all 
£1,202 EL with 11 < x2, f(x1) = f(a2). 
PROOF. a) We go by contraposition: suppose that f is not increasing: then 
there exist 21,72 € I with 71 < x2 such that f(x) > f(x2). Apply the Mean Value 
Theorem to f on [x1, #9]: there exists 71 < c¢ < x2 such that 


f (v2) = f(t 

fo = Led=fen 
L2— Ly 

b) Again, we argue by contraposition: suppose that f is not weakly increasing: 


then there exist 71,22 € I with x1 < x2 such that f(rv1) > f(x2). Apply the Mean 
Value Theorem to f on [21,2]: there exists 71 < ¢ < x2 such that 


f'(o) = f (x2) — f(x1) 


tg — © 
c),d) We leave these proofs to the reader. One may either proceed exactly as in 
parts a) and b), or reduce to them by multiplying f by —1. 


<0. 


<0. 


COROLLARY 5.21 (Zero Velocity Theorem). Let f : I > R be a differentiable 
function with identically zero derivative. Then f is constant. 


PRooF. Since f’(x) > 0 for all « € I, f is weakly increasing on I: x1 < 
tq = > f(x) < f(x2). Since f’(x) < 0 for all x € I, f is weakly decreasing on 
I: v2) <atq = > f(a) > f(x2). But a function which is weakly increasing and 
weakly decreasing satsifies: for all 21 < vo, f(x1) < f(x2) and f(x1) > f(x2) and 
thus f(x1) = f(x): f is constant. 


Above, we deduced Corollary 5.21 from Theorem 5.20. If we had argued directly 
from the Mean Value Theorem the proof would have been shorter: try it! 


COROLLARY 5.22. Suppose f,g: I — R are both differentiable and such that 
f' =9' (equality as functions, i.e., f'(x) = g(a) for alla € I). Then there is a 
constant C € R such that f=g+C, ie. for alla El, f(x) =g(«)+C. 

Proor. Let h = f—g. Then h’ = (f —g)' = f’ —g' =0, so by Corollary 5.21, 
h=C and thus f=g+h=g9+C. 


Corollary 5.22 can be viewed as a uniqueness theorem for differential equations. Let 
f : 1 —R be a function, and consider the set of all functions F : J > R such that 
F’ = f. Then Corollary 5.22 asserts that if there is a function F’ such that F’ = f, 
then there is a one-parameter family of such functions, and more specifically 
that the general such function is of the form F+C. 


On the other hand, the existence question lies deeper: namely, given f : J > R, 
must there exist F : J + R such that F’ = f? In general the answer is no. 


EXERCISE 5.11. Let f: R->R by f(x) =0 for x <0 and f(x) =1 for x > 0. 
Show that there is no function F :R—R such that F’ = f. 


In other words, not every function f : R — R has an antiderivative, i.e., is the 
derivative of some other function. It turns out that every continuous function has 
an antiderivative: this will be proved later. (Much more subtly, there are also some 
discontinuous functions that have antiderivatives...) 
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COROLLARY 5.23. Let f : I > R be a function whose nth derivative f™ is 
identically zero. Then f is a polynomial function of degree at most n—1. 


EXERCISE 5.12. Prove Corollary 5.23. (Hint: use induction.) 


The setting of the Increasing Function Theorem is that of a differentiable function 
defined on an open interval J. This is just a technical convenience: for continu- 
ous functions, the increasing / decreasing / weakly increasing / weakly decreasing 
behavior on the interior of J implies the same behavior at an endpoint of J. 


THEOREM 5.24. Let f : [a,b] > R be continuous at x =a andx=b. 
a) If f is weakly increasing on (a,b), it is weakly increasing on [a, b]. 
b) If f ts incereasing on (a,b), it is increasing on [a, b]. 


PROOF. Step 1: Suppose that f is continuous at a and weakly increasing on 
(a,b). We will show that f is weakly increasing on [a,b). Indeed, assume not: then 
there exists xo € (a,b) such that f(a) > f(xo). Now take e = f(a) — f (xo); since 
f is (right-)continuous at a, there exists 6 > 0 such that for alla <4 < a+, 
|f(a) — f(a)| < f(a) — f(vo), which implies f(z) > f(xo). By taking a < x < 20, 
this contradicts the assumption that f is weakly increasing on (a, b). 

Step 2: Suppose that f is continuous at a and increasing on (a,b). We will show 
that f is increasing on [a, b). Note first that Step 1 applies to show that f(a) < f(x) 
for all x € (a,b), but we want slightly more than this, namely strict inequality. So, 
seeking a contradiction, we suppose that f(a) = f(vo) for some zo € (a,b). But 
now take x € (a, Zo): since f is increasing on (a,b) we have f(x1) < f(%o) = f(a), 
contradicting the fact that f is weakly increasing on [a, b). 

Step 3: In a similar way one can handle the right endpoint b. Now suppose that 
f is increasing on [a,b) and also increasing on (a, }]. It remains to show that f is 
increasing on [a,b]. The only thing that could go wrong is f(a) > f(b). To see that 
this cannot happen, choose any c € (a,b): then f(a) < f(c) < f(b). 


EXERCISE 5.13. Show that we may replace each instance of “increasing” in 
Theorem 5.24 with “decreasing” (and still get a true statement!). 


THEOREM 5.25 (Second Monotone Function Theorem). Let f : I > R be a 
function which is continuous on I and differentiable on the interior I° of I (i.e., 
at every point of I except possibly at endpoints of I, if any). 

a) The following are equivalent: 
(i) f is monotone. 

(ii) Either we have f’(x) > 0 for alla € I° or f'(x) <0 for allx ET. 

b) Suppose f is monotone. The following are equivalent: 

(i) f is not increasing or decreasing. 

(ii) There exist a,b € I° with a < b such that the restriction of f to [a,b] 
is constant. 

(iii) There exist a,b € I° with a < b such that f'(x) =0 for all x € [a,b]. 


Proor. Throughout the proof we restrict our attention to increasing / weakly 
increasing functions, leaving the other case to the reader as a routine exercise. 
a) (i) => (ii): Suppose f is weakly increasing on J. We claim f’(x) > 0 for all 
x €I°. If not, there is a € I° with f’(a) < 0. Then f is decreasing at a, so there 
exists b > a with f(b) < f(a), contradicting the fact that f is weakly decreasing. 
(ii) ==> (i): Immediate from the Increasing Function Theorem and Theorem 5.24. 
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b) (i) == (ii): Suppose f is weakly increasing on J but not increasing on I. By 
Theorem 5.24 f is still not increasing on J°, so there exist a,b € I° witha < b 
such that f(a) = f(b). Then, since f is weakly increasing, for all c € [a, b] we have 
f(a) < f(o) < f(b) = f(a), so f is constant on {a, J. 

(ii) = > (iii): If f is constant on [a,b], f’ is identically zero on [a, }]. 

(iii) ==> (i): If f’ is identically zero on some subinterval [a, b], then by the Zero 
Velocity Theorem f is constant on [a,b], hence is not increasing. 


The next result follows immediately. 


COROLLARY 5.26. Let f : I > R. Suppose f’(x) > 0 for all x € I, and 
f(x) > 0 except at a finite set of points x1,...,%n. Then f is increasing on I. 


EXAMPLE 5.9. A typical application of Theorem 5.26 is to show that the func- 
tion f: RR by f(x) = 2° is increasing on all of R. Indeed, f'(x) = 3x? which 
is strictly positive at allx £0 and0 atx =0. 


5.2. The First Derivative Test. 


We can use Theorem 5.24 to quickly derive another staple of freshman calculus. 


THEOREM 5.27 (First Derivative Test). Let I be an interval, a an interior 
point of I, and f : I + R a function. We suppose that f is continuous on I and 
differentiable on I \ {a}. Then: 

a) If there is 6 > 0 such that f(x) is negative on (a—06,a) and is positive on 
(a,a+06). Then f has a strict local minimum at a. 

b) If there is 6 > 0 such that f’(x) is positive on (a—06,a) and is negative on 
(a,a+6). Then f has a strict local maximum at a. 


PROOF. a) By the First Monotone Function Theorem, since f’ is negative on 
the open interval (a—4, a) and positive on the open interval (a,a+06) f is decreasing 
on (a — 6,a) and increasing on (a,a +). Moreover, since f is differentiable on its 
entire domain, it is continuous at a—06, a and a+, and thus Theorem 5.24 applies 
to show that f is decreasing on [a — 6,a] and increasing on [a,a+ 6]. This gives 
the desired result, since it implies that f(a) is strictly smaller than f(x) for any 
x € [a — 6,a) or in (a,a+ 0]. 

b) As usual this may be proved either by revisiting the above argument or deduced 
directly from the result of part a) by multiplying f by —1. 


Remark: This version of the First Derivative Test is a little stronger than the 
familiar one from freshman calculus in that we have not assumed that f’(a) = 0 
nor even that f is differentiable at a. Thus for instance our version of the test 
applies to f(x) = |x| to show that it has a strict local minimum at ¢ = 0. 


5.3. The Second Derivative Test. 
THEOREM 5.28 (Second Derivative Test). Let a be an interior point of an 
interval I, and let f: 1+ R. We suppose: 
(i) The function f is twice differentiable at a, and 
(ii) We have f'(a) =0. 
Then if f’(a) > 0, f has a strict local minimum at a, whereas if f’(a) <0, f has 
a strict local maximum at a. 
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Proor. As usual it suffices to handle the case f(a) > a. 
Notice that the hypothesis that f is twice differentiable at a implies that f is 
differentiable on some interval (a—46,a+6) (otherwise it would not be meaningful to 
talk about the derivative of f’ at a). Our strategy will be to show that for sufficiently 
small 6 > 0, f(x) is negative for x € (a — 6,a) and positive for x € (a,a+ 4) and 
then apply the First Derivative Test. To see this, consider 
f"(a) hin f(z) mm f'(a) = lim f'(x) ; 


ra x-a rtazr—-a 


We are assuming that this limit exists and is positive, so that there exists 6 > 0 
such that for all x € (a—6,a)U(a,a+0), f @) is positive. And this gives us exactly 


x 


what we want: suppose x € (a —6,a). Then i) >Oand c—a<0,s0 f’(x) < 0. 


a 


On the other hand, suppose x € (a,a +0). Then fe > 0 and  —a > 0, so 


f(z) > 0. So f has a strict local minimum at a by the First Derivative Test. 


Remark: When f’(a) = f(a) = 0, no conclusion can be drawn about the local 
behavior of f at a: it may have a local minimum at a, a local maximum at a, be 
increasing at a, decreasing at a, or none of the above. 


5.4. Sign analysis and graphing. 


When one is graphing a function f, the features of interest include number and 
approximate locations of the roots of f, regions on which f is positive or negative, 
regions on which f is increasing or decreasing, and local extrema, if any. For these 
considerations one wishes to do a sign analysis on both f and its derivative f’. 


Let us agree that a sign analysis of a function g : I — R is the determina- 
tion of regions on which g is positive, negative and zero. 


The basic strategy is to determine first the set of roots of g. As discussed before, 
finding exact values of roots may be difficult or impossible even for polynomial 
functions, but often it is feasible to determine at least the number of roots and 
their approximate location (certainly this is possible for all polynomial functions, 
although this requires justification that we do not give here). The next step is to 
test a point in each region between consecutive roots to determine the sign. 


This procedure comes with two implicit assumptions. Let us make them explicit. 


The first is that the roots of f are sparse enough to separate the domain J into 
“regions”. One precise formulation of of this is that f has only finitely many roots 
on any bounded subset of its domain. This holds for all the elementary functions we 
know and love, but certainly not for all functions, even all differentiable functions: 
we have seen that things like x” sin(+) are not so well-behaved. But this is a con- 
venient assumption and in a given situation it is usually easy to see whether it holds. 


The second assumption is more subtle: it is that if a function f takes a posi- 
tive value at some point a and a negative value at some other point b then it must 
take the value zero somewhere in between. Of course this does not hold for all 
functions: it fails very badly, for instance, for the function f which takes the value 
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1 at every rational number and —1 at every irrational number. 
Let us formalize the desired property and then say which functions satisfy it. 


A function f : J > R has the intermediate value property if for all a,b € I 
with a < b and all L in between f(a) and f(b) —ie., with f(a) < L < f(b) or 
f(b) < L < f(a) — there exists some c € (a,b) with f(c) = L. 


Thus a function has the intermediate value property when it does not “skip” values. 


Here are two important theorems, each asserting that a broad class of functions 
has the intermediate value property. 


THEOREM 5.29. (Intermediate Value Theorem) Let f : [a,b] + R be continu- 
ous. Then f has the intermediate value property. 


Example of a continuous function f : [0,2]g — Q failing the intermediate value 
property. Let f(x) be —1 for 0< x < V2 and f(x) =1 for V2 <2 <1. 


The point of this example is to drive home the point that the Intermediate Value 
Theorem is the second of our three “hard theorems” in the sense that we have no 
chance to prove it without using special properties of the real numbers beyond the 
ordered field axioms. And indeed we will not prove IVT right now, but we will use 
it, just as we used but did not yet prove the Extreme Value Theorem. (However we 
are now not so far away from the point at which we will “switch back”, talk about 
completeness of the real numbers, and prove the three hard theorems.) 


The Intermediate Value Theorem (IVT) is ubiquitously useful. Such innocuous 
statements as every non-negative real number having a square root contain an im- 
plicit appeal to IVT. Further, IVT justifies the following observation. 

Let f : J — R be a continuous function, and suppose that there are only finitely 
many roots, i.e., there are 71,...,%» € J such that f(x;) =0 for alli and f(x) 40 
for all other « € J. Then J\ {x1,...,@n} is a finite union of intervals, and on each 
of them f has constant sign: it is either always positive or always negative. 


So this is how sign analysis works for a function f when f is continuous — a very 
mild assumption. But as above we also want to do a sign analysis of the derivative 
f’: how may we justify this? 


Well, here is one very reasonable justification: if the derivative f’ of f is itself 
continuous, then by IVT it too has the intermediate value property and thus, at 
least if f’ has only finitely many roots on any bounded interval, sign analysis is 
justified. This brings up the following basic question. 


QUESTION 5.30. Let f: 1 > R be a differentiable function. Must its derivative 
f': IR be continuous? 


Let us first pause to appreciate the subtlety of the question: we are not asking 
whether f differentiable implies f continuous: we well know this is the case. Rather 
we are asking whether the new function f’ can exist at every point of J but fail to 
itself be a continuous function. In fact the answer is yes. 
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Example: Let f(x) = x*sin(4). I claim that f is differentiable on all of R but 
that the derivative is discontinuous at x = 0, and in fact that lim,_,9 f’(x) does 
not exist. ... 


THEOREM 5.31. (Darboux) Let f : I > R be a differentiable function. Suppose 
that we have a,b € I with a < b and f'(a) < f'(b). Then for every L € R with 
f'(a) < L < f'(b), there exists c € (a,b) such that f'(c) = L. 


PROOF. Step 1: First we handle the special case L = 0, which implies f’(a) < 0 
and f(b) > 0. Now f is a differentiable — hence continuous — function defined on 
the closed interval [a,b] so assumes its minimum value at some point c € [a,b]. If c 
is an interior point, then as we have seen, it must be a stationary point: f’(c) = 0. 
But the hypotheses guarantee this: since f’(a) < 0, f is decreasing at a, thus 
takes smaller values slightly to the right of a, so the minimum cannot occur at a. 
Similarly, since f’(b) > 0, f is increasing at b, thus takes smaller values slightly to 
the left of 6, so the minimum cannot occur at b. 

Step 2: We now reduce the general case to the special case of Step 1 by defining 
g(x) = f(x) — La. Then g is still differentiable, g’(a) = f’(a) — L < 0 and g’(b) = 
f'(b) — L > 0, so by Step 1, there exists c € (a,b) such that 0 = g’(c) = f’(c) — L. 
In other words, there exists c € (a,b) such that f’(c) = L. 


Remark: Of course there is a corresponding version of the theorem when f(b) < 
L < f(a). Darboux’s Theorem also often called the Intermediate Value The- 
orem For Derivatives, terminology we will understand better when we discuss 
the Intermediate Value Theorem (for arbitrary continuous functions). 


Exercise: Let a be an interior point of an interval IJ, and suppose f : I > R is 
a differentiable function. Show that the function f’ cannot have a simple dis- 
continuity at « = a. (Recall that a function g has a simple discontinuity at a 
if lim, _.a- g(x) and lim, _,,+ g(x) both exist but either they are unequal to each 
other or they are unequal to g(a).) 


5.5. A Theorem of Spivak. 


The following theorem is taken directly from [S, Ch. 11, Thm. 7]. It does not 
seem to be nearly as well known as Darboux’s Theorem (and in fact I think I 
encountered it for the first time in Spivak’s book). 


THEOREM 5.32. Let a be an interior point of I, and let f : I > R. Suppose: 
(i) f ts continuous on I, 
(ii) f is differentiable on I \ {a}, i.e., at every point of I except possibly at a, 
(iii) lima 5a f’(x) = L exists. 
Then f is differentiable at a and f'(a) = L. 


PRooF. Choose 6 > 0 such that (a—6,a+6) C I. Let x € (a,a+64). Then 
f is differentiable at x, and we may apply the Mean Value Theorem to f on [a, 2]: 
there exists cz € (a,x) such that 
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Now, as  — a every point in the interval [a,2] gets arbitrarily close to x, so 
lim;-+¢ Cy = x and thus 


Pie Sin 2 es Pe Ske POSE 


raat r—-a a—at asat 


By a similar argument involving x € (a — 6,a) we get 
(a) — Via fl) — 
fi,(a) = lim (2) =, 
so f is differentiable at a and f’(a) = L. 


6. Inverse Functions I: Theory 


6.1. Review of inverse functions. 


Let X and Y be sets, and let f : X — Y be a function. An inverse function is a 
function g: Y — X such that 


gof=1lx:X 3X, fog=ly:Y-Y. 
Let’s unpack this notation: it means the following: first, that 
for all x € X, (go f)(x) = g(f(@)) =a 
and second, that 


for all y € Y,(f og)(y) = f(g(y)) = ¥. 


PROPOSITION 5.33 (Uniqueness of Inverse Functions). Let f : X > Y be a 
function. Suppose that 91,92: Y — X are both inverses of f. Then gi = go. 


PrRoorF. For all y € Y, we have 
gi(y) = (92° F)(91(y)) = 92(F (gy) = ga(y)- 


Now that we know that the inverse function to f is always unique provided it exists, 
we denote it by f~!. (Caution: this has nothing to do with rE eg. sin (x) # 


esc(z) = =+..) We turn next to giving conditions for the existence of the inverse 
function. Recall that f : X > Y is injective if 
for all x1,%2 € X, 71 4 XQ f(ai) 4 f (a2). 


In other words, distinct x-values get mapped to distinct y-values. Also f : X > Y 
is surjective if 
for all y € Y, there exists at least one « € X such that y = f(z). 


Putting these two concepts together we get the important notion of a bijective 
function f : X — Y,i.e., a function which is both injective and surjective. Other- 
wise put, for all y € Y there exists exactly one x € X such that y = f(x). It may 
well be intuitively clear that bijectivity is exactly the condition needed to guarantee 
existence of the inverse function: if f is bijective, we define f~!(y) = xy, the unique 
element of X such that f(z,) = y. And if f is not bijective, this definition breaks 
down and thus we are unable to define f~!. Nevertheless we ask the reader to bear 
with us as we give a slightly tedious formal proof of this. 


THEOREM 5.34 (Existence of Inverse Functions). For a function f: X > Y, 
the following are equivalent: 


(i) The function f is bijective. 
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(ii) The function f admits an inverse function. 


PRoorF. (i) = > (ii): If f is bijective, then as above, for each y € X there 
exists exactly one element of X — say x, — such that f(x,) = y. We may therefore 
define a function g : Y — X by g(y) = ay. Let us verify that g is in fact the inverse 
function of f. For any x € X, consider g(f(x)). Because f is injective, the only 
element x’ € X such that f(a’) = f(x) is 2’ = x, and thus g(f(x)) = z. For any 
y €Y, let x, be the unique element of X such that f(z,) = y. Then 


f(gy)) = f(ty) = y- 
(ii) => (i): Suppose that f—1 exists. To see that f is injective, let x1,22 € X be 
such that f(r1) = f(x2). Applying f~+ on the left gives 
eT Fey = FFs) = m- 


So f is injective. To see that f is surjective, let y¢ Y. Then f(f~'(y)) = y, so 
there is x € X with f(x) = y, namely « = f—1(y). 


For any function f : X — Y, we define the image of f to be 
{y € Y |x eX such that y = f(x)}. 
The image of f is often denoted f(X).° 


We now introduce the trick of codomain restriction. Let f : X — Y bea 
function. If we replace the codomain Y by the image f(X), we still get a well- 
defined function f : X + f(X), and this new function is tautologically surjective.® 


EXAMPLE 5.10. Let f : R > R by f(x) = 27. Then f(R) = [0,00), and 
although x? :R — R is not surjective, x? :R — [0,00) certainly is. 


Since a codomain-restricted function is always surjective, it has an inverse iff it is 
injective iff the original function is injective. Thus: 


COROLLARY 5.35. For a function f : X > Y, the following are equivalent: 


(i) The codomain-restricted function f : X + f(X) has an inverse function. 
(ii) The original function f is injective. 


6.2. The Interval Image Theorem. 


Next we want to return to earth by considering functions f : J — R and their 
inverses, concentrating on the case in which f is continuous. 


THEOREM 5.36 (Interval Image Theorem). Let J Cc R be an interval, and let 
f:I-R be a continuous function. Then the image f(I) of f is also an interval. 


ProoF. For now we will give the proof when I = [a,}], i.e., is closed and 
bounded. The general case will be discussed later. 

Suppose f : [a,b] > R is continuous. Then f has a minimum value m, say at 
Lm and a maximum value M, say at x,y. Thus the image f([a,}]) of f is a subset 


5This is sometimes called the range of f, but sometimes not. It is safer to call it the image! 

STmagine that you manage the up-and-coming band Yellow Pigs. You get them a gig one 
night in an enormous room filled with folding chairs. After everyone sits down you remove all the 
empty chairs, and the next morning you write a press release saying that Yellow Pigs played to a 
“packed house.” This is essentially the same dirty trick as codomain restriction. 
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of [m, M]. Moreover, if L € (m,M), then by the Intermediate Value Theorem there 
exists c in between x, and xy such that f(c) = L. So f([a,b]) = [m, M]. 


Exercise: Let J be a nonempty interval which is not of the form [a,b]. Let J be any 
nonempty interval. Show: there is a continuous function f : J > R with f(J) = J. 


6.3. Monotone Functions and Invertibility. 


Recall f : I — R is strictly monotone if it is either increasing or decreasing. 
Every strictly monotone function is injective. Therefore our dirty trick of codomain 
restriction works to show that if f : J > R is strictly monotone, f : J > f(J) is 
bijective, hence invertible. Thus in this sense we may speak of the inverse of any 
strictly monotone function. 


PROPOSITION 5.37. Let f: I > f(L) be a strictly monotone function. 
a) If f is increasing, then f—1 : f(I) 3 I is increasing. 
b) If f is decreasing, then f—1: f(I) 3 I is decreasing. 


Proor. As usual, we will content ourselves with the increasing case, the de- 
creasing case being so similar as to make a good exercise for the reader. 

Seeking a contradiction we suppose that f~! is not increasing: that is, there 
exist y1 < yo € f(Z) such that f—'(y,) is not less than f~!(y2). Since f~+ is an 
inverse function, it is necessarily injective (if it weren’t, f itself would not be a 
function!), so we cannot have f~!(y,) = f~!(y2), and thus the possibility we need 
to rule out is f~!(ye) < f7t(y1). But if this holds we apply the increasing function 
f to get yo = f(f-*(y2)) < f(f74(y1)) = m1, a contradiction. 


LEMMA 5.38. (A-V Lemma) Let f:I—R. The following are equivalent: 
(i) f ts not monotone: i.e., f is neither increasing nor decreasing. 
(ii) At least one of the following holds: 
(a) f is not injective. 
(b) f admits a A-configuration: there exista <b<cE€I with f(a) < f(b) > fle). 
(c) f admits a V-configuration: there exista <b <c€I with f(a) > f(b) < fle). 


EXERCISE 5.14. Prove Lemma 5.88. 
THEOREM 5.39. If f : I > R is continuous and injective, it is monotone. 


PRooF. We will suppose that f is injective and not monotone and show that 
it cannot be continuous, which suffices. We may apply Lemma 5.14 to conclude 
that f has either a A configuration or a V configuration. 

Suppose first f has a A configuration: there exist a < b <c€ I with f(a) < 
f(b) > f(c). Then there exists L € R such that f(a) < L < f(b) > L > f(c). lf f 
were continuous then by the Intermediate Value Theorem there would be d € (a,b) 
and e € (b,c) such that f(d) = f(e) = L, contradicting the injectivity of f. 

Next suppose f has a V configuration: there exist a < b < c € I such that 
f(a) > f(b) < f(c). Then there exists Z € Rsuch that f(a) > L > f(b) <L< fice). 
If f were continuous then by the Intermediate Value Theorem there would be 
d € (a,b) and e € (b,c) such that f(d) = f(e) = L, contradicting injectivity. 
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6.4. Inverses of Continuous Functions. 


THEOREM 5.40. (Continuous Inverse Function Theorem) Let f : I > R be 
injective and continuous. Let J = f(1) be the image of f. 
a) f: 1 J is a bijection, and thus there is an inverse function f~+: J > I. 
b) J is an interval in R. 
c) If I = [a,b], then either f is increasing and J = [f(a), f(b)] or f is decreasing 
and J = (f(b), f(a)] 


d) The function f-+: J > I is also continuous. 


ProoF. [S, Thm. 12.3] Parts a) through c) simply recap previous results. The 
new result is part d), that f~1 : J + I is continuous. By part c) and Proposition 
5.37, either f and f~! are both increasing, or f and f~! are both decreasing. As 
usual, we restrict ourselves to the first case. 

Let b € J. We must show that lim,_,, f~'(y) = f~'(b). We may write b = f(a) 
for a unique a € J. Fix « > 0. We want to find 6 > 0 such that if f(a) -d <y< 
f(a) +6, then a—e< f7l(y) <ate. 

Take 6 = min(f(a+ ¢) — f(a), f(a) — f(a—)). Then: 

f(a—«) < fla) —4, fla) +6 < fate), 
and thus if f(a) — 6 < y < f(a) + 6 we have 
fla—6) < f(a)-—d<y< f(a) +6 < flate). 


Since f—! is increasing, we get 


f(Fla-6)) < fy) < fF (F(at+ 9), 


or 


IG) See foes Abe 


To be honest, I don’t find the above proof very enlightening. After reflecting on my 
dissatisfaction with it, I came up with an alternate proof that I find conceptually 
simpler, but which depends on the Monotone Jump Theorem, a characteri- 
zation of the possible discontinuities of a monotone function. The proof uses the 
completeness of the real numbers, so is postponed to the next chapter. 


6.5. Inverses of Differentiable Functions. 


In this section our goal is to determine conditions under which the inverse f~! 
of a differentiable funtion is differentiable, and if so to find a formula for (f~1)’. 


Let’s first think about the problem geometrically. The graph of the inverse func- 
tion y = f—'(a) is obtained from the graph of y = f(x) by interchanging x and 
y, or, put more geometrically, by reflecting the graph of y = f(a) across the line 
y = x. Geometrically speaking y = f(x) is differentiable at x iff its graph has 
a well-defined, nonvertical tangent line at the point (x, f(x)), and if a curve has 
a well-defined tangent line, then reflecting it across a line should not change this. 
Thus it should be the case that if f is differentiable, so is f~!. Well, almost. Notice 
the occurrence of “nonvertical” above: if a curve has a vertical tangent line, then 
since a vertical line has “infinite slope” it does not have a finite-valued derivative. 
So we need to worry about the possibility that reflection through y = x carries a 
nonvertical tangent line to a vertical tangent line. When does this happen? Well, 
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the inverse function of the straight line y = ma-+0 is the straight line y = 4 (x —b) 
—i.e., reflecting across y = x takes a line of slope m to a line of slope 2. Morever, 
it takes a horizontal line y = c to a vertical line x = c, so that is our answer: at 
any point (a,b) = (a, f(a)) such that f’(a) = 0, then the inverse function will fail 
to be differentiable at the point (b,a) = (b, f~1(b)) because it will have a vertical 
tangent. Otherwise, the slope of the tangent line of the inverse function at (b, a) is 


precisely the reciprocal of the slope of the tangent line to y = f(a) at (a,b). 


Well, so the geometry tells us. It turns out to be quite straightforward to adapt 
this geometric argument to derive the desired formula for (f~1)'(b), under the as- 
sumption that f is differentiable. We will do this first. Then we need to come back 
and verify that indeed f~! is differentiable at b if f’(f~1(b)) exists and is nonzero: 
this turns out to be a bit stickier, but we are ready for it and we will do it. 
PROPOSITION 5.41. Let f : I > J be bijective and differentiable. Suppose 
that the inverse function f—!: J + I is differentiable at b € J. Then 
1 

‘ie 4) 

PO" FEB) 
In particular, if f~+ is differentiable at b then f'(f—+(b)) 40. 


PROOF. We need only implicitly differentiate the equation 


fF (2) = 2, 


getting 

(21) (f-*)' (F(a) f(a) = 1, 

or re j 
(fF (x) = Fa) 


To apply this to get the derivative at b € J, we just need to think a little about our 
variables. Let a = f—1(b), so f(a) =. Evaluating the last equation at x = a gives 


ein _ 1 7 1 
(PO) = Fay = FO) 


Moreover, since by (21) we have (f~1)'(b) f’(f-1(b)) = 1, f’(f-1(0)) 4 0. 


As mentioned above, unfortunately we need to work a little harder to show the 
differentiability of f~', and for this we cannot directly use Proposition 5.41 but 
end up deriving it again. Well, enough complaining: here goes. 


THEOREM 5.42 (Differentiable Inverse Function Theorem). Let f : I — J be 
continuous and bijective. Let b be an interior point of J and put a = f—*(b). 
Suppose that f is differentiable at a and f'(a) 40. Then f—' is differentiable at b, 
with the familiar formula 
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Since J = f(J), every b+h € J is of the form 
b+h= f(a+kn) 


for a unique kp, € I.” Since b+h= f(at+kp), f-'(b +h) =a+ ky; let’s make this 
substitution as well as h = f(a+k,) — f(a) in the limit we are trying to evaluate: 
Sy ee oe an! Kn a 1 
Oe) GER Ko EO fGrk) iG) tiene 


kp, 


We are getting close: the limit now looks like the reciprocal of the derivative of f 
at a. The only issue is the pesky kp, but if we can show that limp_.o kp = 0, then 
we may simply replace the “limp_,9” with “lim,, 9” and we’ll be done. 
But k, = f7!(b +h) —a, so — since f~! is continuous by Theorem 5.40 — we 
have 
lim k, = lim f~'(b+ h) —a = f-1(6+0) -a= f-'(b) -a=a—-a=0. 
h-0 h-0 
So ash > 0, ky, — 0 and thus 
1 1 1 


(f~)'() =  LetI=FOD ~ F(a) FFB) 


lim,,, + kn 


7. Inverse Functions II: Examples and Applications 


7.1. an. 


Let n be a positive integer. In this section we illustrate the preceding concepts 
by defining and differentiating the nth root function am. The reader should not 
now be surprised to hear that we give separate consideration to the cases of odd n 
and even n. 


Either way, let n > 1 be an integer, and consider 

f: ROR, tH". 
Case 1: n = 2k+1 is odd. Then f’(x) = (2k+1)x?* = (2k+1)(zx")? is non-negative 
for all « € R and not identically zero on any subinterval [a,b] with a < b, so by 
Theorem 5.25 f : R > R is increasing. Moreover, we have limz,+.. f(a) = +00. 
Since f is continuous, by the Intermediate Value Theorem the image of f is all of 


R. Moreover, f is everywhere differentiable and has a horizontal tangent only at 
x =0. Therefore there is an inverse function 


f-':ROR 


which is everywhere continuous and differentiable at every x € R except x = 0. It 
is typical to call this function xn. 


Case 2: n = 2k is even. Then f’(x) = (2k)x?*~1 is positive when 2 > 0 and 
negative when x < 0. Thus f is decreasing on (—oo, 0] and increasing on [0, 00). In 
particular it is not injective on its domain. If we want to get an inverse function, 
we need to engage in domain restriction. Unlike codomain restriction, which can 


7Unlike Spivak, we will include the subscript k, to remind ourselves that this k is defined in 
terms of h: to my taste this reminder is worth a little notational complication. 
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be done in exactly one way so as to result in a surjective function, domain restric- 
tion brings with it many choices. Luckily for us, this is a relatively simple case: if 
D CR, then the restriction of f to D will be injective if and only if for each x € R, 
at most one of x,—a lies in D. If we want the restricted domain to be as large as 
possible, we should choose the domain to include 0 and exactly one of x,—ax for 
all x > 0. There are still lots of ways to do this, so let’s try to impose another 
desirable property of the domain of a function: namely, if possible we would like 
it to be an interval. A little thought shows that there are two restricted domains 
which meet all these requirements: we may take D = [0,00) or D = (—ow, 0}. 


7.2. L(x) and E(x). 


Consider the function | : (0,00) > R given by I(x) = 4. As advertised, we will 
soon be able to prove that every continuous function has an antiderivative, so bor- 
rowing on this result we define L : (0,00) > R to be such that L’(#) = I(x). More 
precisely, recall that when they exist antiderivatives are unique up to the addition 
of a constant, so we may uniquely specify L(x) by requiring L(1) = 0. 


PROPOSITION 5.43. For all x,y € (0,00), we have 
(22) L(xy) = L(x) + L(y). 
Proor. Let y € (0,00) be regarded as fixed, and consider the function 
f(a) = L(xy) — L(x) — L(y). 


We have 


fi (x) = Ly) (ey)! = Lx) = y= 4-2 =o, 


xy oe ty. ae 


By the zero velocity theorem, the function f(a) is a constant (depending, a priori 
on y), say Cy. Thus for all x € (0, 00), 


L(ay) = L(x) + L(y) + Cy. 
If we plug in x = 1 we get 
and thus Cy = 0, so L(xy) = L(x) + L(y). 


CoROLLARY 5.44. a) For all x € (0,00) andn € Z*, we have L(x") = nL (2). 
b) For x € (0,00), we have L(+) = —L(z). 
c) We have limz-+00 L(x) = 00, limy_.9+ L(x) = —o0. 


d) We have L((0,0c0)) = R. 


PROOF. a) An easy induction argument using L(a?) = L(a) + L(x) = 2L (a). 
b) For any « € (0,00) we have 0 = L(1) = L(x- +) = L(x) + L(4). 
c) Since L'(x) = 4 > 0 for all x € (0,00), L is increasing on (0,00). Since 
L(1) = 0, for any « > 0, L(x) > 0. To be specific, take C = L(2), so C > 0. 
Then by part a), L(2”) = nL(2) = nC. By the Archimedean property of R, this 
shows that L takes arbitaririly large values, and since it is increasing, this implies 
limg-+00 L(x) = oo. To evaluate lim,_,9+ L(a) we may proceed similarly: by part 
b), L($) = —L(2) = -C < 0, so L(x) = —nL(2) = —Cn, so L takes arbitrarily 
small values. Again, combined with the fact that LZ is increasing, this implies 
lim,_,9+ L(x) = —oo. (Alternately, we may evaluate lim,_,9+ L(a) by making the 
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change of variable y = 4 and noting that as x 4 0+, y > oo+. This is perhaps 
more intuitive but is slightly tedious to make completely rigorous.) 

d) Since L is differentiable, it is continuous, and the result follows immediately from 
part c) and the Intermediate Value Theorem. 


Definition: We define e to be the unique positive real number such that L(e) = 1. 
(Such a number exists because L : (0,00) — R is increasing — hence injective and 
has image (—oo, 00). Thus in fact for any real number a there is a unique positive 
real number £ such that L(3) = a.) 


Since L(x) is everywhere differentiable with nonzero derivative 1, the differentiable 
inverse function theorem applies: L has a differentiable inverse function 
E:R- (0,00), E(0) =1. 
Let’s compute LE’: differentiating L(E(x)) = x gives 
_ E') 


1= L (Be) Ee) = By 


In other words, we get 


COROLLARY 5.45. For all x,y € R we have E(x + y) = E(x)E(y). 


ProoF. To showcase the techniques available, we give three different proofs. 
First proof: For y € R, let E,(z) = E(x+y). Put f(z) = ae Then 


(x) = eB (@) ~ By) Hla) _ Ele +p) E(e) ~ Be + le + 9 EC) 
E(x)? E(a)? 
_ Ee +y)E(x) — Ee +y)-1- Ea) _ 
E(a)? , 
By the Zero Velocity Theorem, there is C, € R such that for all « € R, f(x) = 
E(a+y)/E(2) =Cy, or E(x + y) = E(x)Cy. Plugging in x = 0 gives 


Ely) = E(0)C(y) =1-Cly) = Cty), 


so 
E(a + y) = E@)Ely). 
Second proof: We have 


= (Fans) = L(E(« +y)) — L(E(@)) - L(E)) =2@+y-2-y=0. 
The unique x € (0,00) such that L(x) = 0 is x = 1, so we must have 
Beto 
E(@)B(y) 


or 
E(x+y) = E(x) Ely). 
Third proof: For any y1, y2 > 0, we have 
L(yry2) = L(y1) + L(y2). 
Put y; = E(a1) and yp = E(x2), so that x1 = L(y1), x2 = L(y2) and thus 
E(a1)E(@2) = yiy2 = E(L(yiy2)) = E(L(y1) + L(y2)) = E(21 + 22). 
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Since & and L are inverse functions and L(e) = 1, we have E(1) = e. Now 
the previous discussion must suggest to any graduate of freshman calculus that 
E(x) = e*: both functions defined and positive for all real numbers, are equal to 
their own derivatives, convert multiplication into addition, and take the value 1 at 
x = 0. How many such functions could there be? 


PROPOSITION 5.46. Let f : R > R be a differentiable function such that f'(x) = 
f(x) for alla ER. There is a constant C such that f(x) = CE(a) for allx ER. 


ProoF. Define a function g: R > R by g(x) = OE Then for all z € R, 


B(x) f(x) — F(a) f(@) _ E(@)f(@) — E@)f@) 
g (x) = Ela? ~ EG? =, 


By the Zero Velocity Theorem g = £ is constant: f(#) = CE(z«) for all x. 


In other words, if there really is a function f(x) = e* out there with f’(a) = e” and 
f (0) =1, then we must have e” = E(x) for all x. The point of this logical maneuver 
is that although in precalculus mathematics one learns to manipulate and graph 
exponential functions, the actual definition of a® for irrational x is not given, and 
indeed I don’t see how it can be given without using key concepts and theorems of 
calculus. But, with the functions E(x) and L(z) in hand, let us develop the theory 
of exponentials and logarithms to arbitrary bases. 


Let a > 0 be a real number. How should we define a*? In the following slightly 
strange way: for any x € R, 

a” := E(L(a)z). 
Let us make two comments: first, if a = e this agrees with our previous definition: 
e* = E(xL(e))) = E(x). Second, the definition is motivated by the following 
desirable law of exponents: (a?)° = a°*. Indeed, assuming this holds unrestrictedly 
for b,c € R and a > 1, we would have 


a” = E(xloga) = erloge — (ele 2)” =e: 
But here is the point: we do not wish to assume that the laws of exponents work 
for all real numbers as they do for positive integers...we want to prove them! 
PROPOSITION 5.47. Fix a € (0,00). For x € R, we define 
a” := E(L(a)z). 
Ifa#1, we define 
] =, 
a) The function a® is differentiable and (a®*)! = L(a)a*. 
b) The function log, « is differentiable and (log, x)! = Taye’ 
c) Supposea>1. Then a® is increasing with image (0, 00), log, x is increas- 
ing with image (—co, co), and a® and log, x are inverse functions. 
) For all x,y ER, we have a?t¥ = ata’. 
e) For alla >0 andy €R, we have (a”)¥ = a. 
) 
) 


For all x,y > 0, we have log, (xy) = log, x + log, y. 
For allz > 0 andy €R, we have log, (x”) = ylog, z. 
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PROOF. a) We have 
(a*)’ = E(L(a)a) = E'(L(a)x)(L(a)x)! = E(L(a)x) - L(a) = L(a)a”. 
b) We have 


L(2)\' 1 

] = = : 

(ozate)! = (F9) = 

c) Since their derivatives are always positive, a” and log, x are both increasing 


functions. Moreover, since a > 1, L(a) > 0 and thus 


lim a” = lim E(L(a)r) = E(w) =~, 
XL Co 


&—00 
| _ 4 Le) _ 00 _ 
jim Jog, (2) = Jim 575) = Tyqy = © 


Thus a® : (—00, 00) > (0,00) and log, x : (0,00) + (—oo, 00) are bijective and thus 

have inverse functions. Thus check that they are inverses of each other, it suffices 

to show that either one of the two compositions is the identity function. Now 
La") _ LEL(a@)x)) _ Lae _ 


= Ta Bay Be) 


d) We have 
a¥ = E(L(a)(w + y)) = E(L(a)a + L(a)y) = B(L(a)x) E(E(a)y) = a". 
e) We have 


f) We have 


ate) sae ay a) De) 
g) We have 
L(x) _ L(EL(z)y)) _ Lle)y 


log, 24 = = = = ylog, x. 


L(a) L(a) L(a) 


Having established all this, we now feel free to write e*” for E(x) and log x for L(a). 


EXERCISE 5.15. Suppose0 <a< 1. Show: a® is decreasing with image (0,00), 
log, x is decreasing with image (0,00), and a® and log, x are inverse functions. 


EXERCISE 5.16. Prove the change of base formula: for all a,b,c > 0 with 
a,c #1, we have 
log. 
log, b = Be 


a 


~ log.a’ 


The following is mostly an example of the use of induction to prove a result in 
differential calculus. 


PROPOSITION 5.48. Let f(x) = e*. Then for alln € Z* there is a polynomial 
P, (ax), of degree n, such that 
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PRooF. By induction on n. 
Base case (n = 1): 


et” — Qe” = Pi(a)e™, where P;(ax) = 2x, a degree one polynomial. 


Inductive step: Assume that for some positive integer n there exists P, (a) of degree 


uu 2 2 n+1 2: 
n such that — e* =P, (a)e* . So ore" — 
d dad” j2m d 


poe IH © 2 _ pl x? 2 (pl x? 
ame age ire Pi(a)je” +2xP,(x)e (P) (x) + 2a@P,(x)) e® . 


Now, since P,,(x) has degree n, P’(x) has degree n — 1 and 2xP,,(x) has degree 
n+1. If f and g are two polynomials such that the degree of f is different from 
the degree of g, then deg(f + g) = max(deg(f), deg(g)). In particular, P,41(x) := 
P! (x) + 2xP,,(x) has degree n + 1, completing the proof of the induction step. 


7.3. Some inverse trigonometric functions. 


We now wish to consider inverses of the trigonometric functions: sine, cosine, tan- 
gent, and so forth. Right away we encounter a problem similar to the case of «” for 
even n: the trigonometric functions are periodic, hence certainly not injective on 
their entire domain. Once again we are forced into the art of domain restriction 
(as opposed to the science of codomain restriction). 


Consider first f(z) = sinz. To get an inverse function, we need to restrict the 
domain to some subset S on which f is injective. As usual we like intervals, and 
the maximal possible length of an interval on which the sine function is injective 
is 7, attained by any interval at which the function either increases from —1 to 1 
or decreases from 1 to —1. This still gives us choices to make. The most standard 
choice — but to be sure, one that is not the only possible one nor is mathemati- 


cally consecrated in any particular way — is to take I = [|", 5]. We claim that 
f is increasing on J. To check this, note that f’(x) = cos is indeed positive on 
(<4, 5). We have f([=*,5]) = [-1,1]. The inverse function here is often called 


arcsinx (“arcsine of x”) in an attempt to distinguish it from x =cscx. This is 


as good a name as any: let’s go with it. We have 
—1 1 
23h 
As the inverse of an increasing function, arcsin x is increasing. Moreover since sin x 


has a nonzero derivative on (=, $), arcsin x is differentiable there. Differentiating 


arcsin : [—1, 1] > [ 


sin(arcsin x)) = a, 
we get 
cos(arcsin x) arcsin’ (x) = 1, 


or 
1 


— arcsin x = ——~—_—-_... 
dx cos(arcsin 2) 

This looks like a mess, but a little trigonometry will clean it up. The key is to 
realize that cosarcsinx means “the cosine of the angle whose sine is «” and that 
there must be a simpler description of this. If we draw a right triangle with angle 
6 = arcsin x, then to get the ratio of the opposite side to the hypotenuse to be x we 


may take the length of the opposite side to be x and the length of the hypotenuse to 
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be 1, in which case the length of the adjacent side is, by the Pythagorean Theorem, 


V1-—x?. Thus cos@ = V1 — 2”, so finally 


; 1 
— arcsin s = ————.. 
Vl—«2? 
Now consider f(#) = cosa. Since f is even, it is not injective on any interval 
containing 0 in its interior. Reflecting a bit on the graph of f(x) = cosa one sees 
that a reasonable choice for the restricted domain is [0,7]: since f’(x) = —sin z is 
negative on (0,7) and 0 and 0 and a, f(a) is decreasing on [0,7] and hence injective 
there. Its image is f([0,7])) = [-1,1]. Therefore we have an inverse function 
arccos : [—1,1] > [0,z]. 


Since cos x is continuous, so is arccosxz. Since cos is differentiable and has zero 
derivative only at 0 and 7, arccos x is differentiable on (—1, 1) and has vertical tan- 
gent lines at = —1 and x = 1. Morever, since cos x is decreasing, so is arccos @. 


We find a formula for the derivative of arccos just as for arcsin: differentiating 
cos arccos 7 = x 
gives 
“ / 
— sin(arccos x) arccos’ 7 = 1, 


or 
—1 


sin arccos x 
Again, this may be simplified. If y = arccosx, then x = cosy, so if we are on the 
unit circle then the y-coordinate is siny = /1 — x?, and thus 

-1 
V1— 2? 
Remark: It is hard not to notice that the derivatives of the arcsine and the arccosine 
are simply negatives of each other, so for all x € [0, 5], 


arccos’ t= 
/ 
arccos v= 


arccos’ x + arcsin’ x = 0. 
By the Zero Velocity Theorem, we conclude 
arccos 7 + arcsinaz = C 
for some constant C. To determine C’, simply evaluate at x7 = 0: 


C = arccos0 + arcsin0 = . +0= = 
and thus for all « € [0, 3] we have 
arccos x + arcsin x = a 


So the angle 6 whose sine is x is complementary to the angle y whose cosine is 2x. 


Finally, consider f(x) = tanz = “2£. The domain is all real numbers for which 
3m 


cosx # 0, so all real numbers except +5,-+53,.... The tangent function is peri- 
odic with period z and also odd, which suggests that, as with the sine function, we 
should restrict this domain to the largest interval about 0 on which f is defined and 


injective. Since f’(x) = sec? « > 0, f is increasing on (=, $) and thus is injective 
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there. Moreover, lim,;+42 tanz = +00, so by the Intermediate Value Theorem 


f((=&.35)) =R. Therefore we have an inverse function 
arctan : R > (F z) : 


Since the tangent function is differentiable with everywhere positive derivative, the 
same is true for arctan x. In particular it is increasing but bounded: lim, +. arctan x = 


+5. In other words the arctangent has horizontal asymptotes at y = +4. 


8. Some Complements 


The Mean Value Theorem and its role in “freshman calculus” has been a popular 
topic of research and debate over the years. 


A short paper of W.J. Knight improves upon the Zero Velocity Theorem. 


THEOREM 5.49. (Right-Handed Zero Velocity Theorem [Kn80]) Let f : [a,b] > 
R be continuous. If the right-hand derivative f(x) exists and is 0 for all x € (a,b), 
then f is constant. 


The proof is modelled upon the usual one: one starts with a right-handed Rolle’s 
Theorem, deduces a right-handed Mean Value Inequality, and then Theorem 5.49. 


CHAPTER 6 


Completeness 


1. Dedekind Completeness 
1.1. Introducing (LUB) and (GLB). 


Gather round, my friends: the time has come to tell what makes calculus work. 


Recall that we began the course by considering the real numbers as a set endowed 
with two binary operations + and - together with a relation <, and satisfying a 
longish list of familiar axioms (PO) through (P12), the ordered field axioms. We 
then showed that using these axioms we could deduce many other familiar proper- 
ties of numbers and prove many other identities and inequalities. 


However we did not claim that (PO) through (P12) was a complete list of axioms 
for R. On the contrary, we saw that this could not be the case: for instance the 
rational numbers Q also satisfy the ordered field axioms but — as we have taken 
great pains to point out — most of the “big theorems” of calculus are meaningful 
but false when regarded as results applied to the system of rational numbers. So 
there must be some further axiom, or property, of R which is needed to prove the 
three Interval Theorems, among others. 


Here it is. Among structures F’ satisfying the ordered field axioms, consider the 
following further property: 


(P14): Least Upper Bound Axiom (LUB): Let S be a nonempty subset of 
F which is bounded above. Then S admits a least upper bound. 


This means exactly what it sounds like, but it is so important that we had better 
make sure. Recall a subset S of F is bounded above if there exists M € R such 
that for all z € S, x < M. (For future reference, a subset S of R is bounded 
below if there exists m € F' such that for all ¢ € S,m <a.) By a least upper 
bound for a subset S' of /, we mean an upper bound M which is less than any 
other upper bound: thus, M is a least upper bound for S if M is an upper bound 
for S and for any upper bound M’ for S$, M < M’. 


There is a widely used synonym for “the least upper bound of S”, namely the 
supremum of S. We also introduce the notation lub S = sup S for the supremum 


of a subset S$ of an ordered field (when it exists). 


The following is a useful alternate characterization of supS: the supremum of 
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S is an upper bound M for S with the property that for any M’ < M, M’ is not 
an upper bound for S: explicitly, for all M7’ < M, there exists x € S with M’ < x. 


The definition of the least upper bound of a subset S makes sense for any set 
X endowed with an order relation <. Notice that the uniqueness of the supremum 
sup S' is clear: we cannot have two different least upper bounds for a subset, be- 
cause one of them will be larger than the other! Rather what is in question is the 
existence of least upper bounds, and (LUB) is an assertion about this. 


Taking the risk of introducing even more terminology, we say that an ordered field 
(F,+,-,<) is Dedekind complete! if it satisfies the least upper bound axiom. 
Now here is the key fact lying at the foundations of calculus and real analysis. 


THEOREM 6.1. a) The ordered field R is Dedekind complete. 
b) Conversely, any Dedekind complete ordered field is isomorphic to R. 


Part b) of Theorem 6.1 really means the following: if F’ is any Dedekind complete 
ordered field then there is a bijection f : F — R which preserves the addition, 
multiplication and order structures in the following sense: for all z,y € F, 


e f(xty) =flx)+ fy), 
© f(vy) = f(a) fly), and 
elfa<y, then f(x) < f(y). 
This concept of “isomorphism of structures” comes from a more advanced course — 
abstract algebra — so it is probably best to let it go for now. One may take part 
b) to mean that there is essentially only one Dedekind complete ordered field: R. 
The proof of Theorem 6.1 involves constructing the real numbers in a mathe- 
matically rigorous way. This is something of a production, and although in some 
sense every serious student of mathematics should see a construction of R at some 
point of her career, this sense is similar to the one in which every serious student 
of computer science should build at least one working computer from scratch: in 
practice, one can probably get away with relying on the fact that many other people 
have performed this task in the past. Spivak does give a construction of R and a 
proof of Theorem 6.1 in the “Epilogue” of his text. And indeed, if we treat this 
material at all it will be at the very end of the course. 


After discussing least upper bounds, it is only natural to consider the “dual” con- 
cept of greatest lower bounds. Again, this means exactly what it sounds like but it 
is so important that we spell it out explicitly: if S is a subset of an ordered field F, 
then a greatest lower bound for S, or an infimum of S, is an element m € F 
which is a lower bound for S$ — i.e., m < x for all x € S — and is such that if m’ 
is any lower bound for S then m’ < m. Equivalently, m = inf S iff m is a lower 
bound for S$ and for any m’ > m there exists x € S' with x < m’. Now consider: 


(P14’): Greatest Lower Bound Axiom (GLB): Let S be a nonempty subset 
of F which is bounded below. Then S admits a greatest lower bound, or infimum. 


lit is perhaps more common to say “complete” instead of “Dedekind complete”. I have my 
reasons for preferring the lengthier terminology, but I won’t trouble you with them. 
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EXAMPLE 6.1. In any ordered field F', we may consider the subset 
Sp={ceF | 2? <2}. 


Then Sp is nonempty and bounded: indeed 0 € Sp and if x € Sr, then |x| < 2. Of 
course in the previous inequality we could do better: for instance, if |x| > 3 then 
x > 3 > 2, so also ore is a lower bound for Sr and 3 is an upper bound for Sr. 
Of course we could do better still... 

Indeed the bounded set Sr will have an infimum and a supremum if and only if 
there are best possible inequalitiesr € S = m<a< WM, ie., for which no 
improvement on either m or M is possible. Whether such best possible inequalities 
exist depends on the ordered field F. Indeed, it is clear that if M = sup Sp exists, 
then it must be a positive element of F with M? = 2: or in other words, what in 
precalculus mathematics one cavalierly writes as M = V/2. similarly, if m = inf Sp 
exists, then it must be a negative element of F with m? = 2, or what we usually 
write as —\/2. But here’s the point: how do we know that our ordered field F 
contains such an element V2? 

The answer of course is that depending on F' such an element may or may not 
exist. As we saw at the beginning of the course, there is no rational number x with 
x? = 2, so if F = Q then our set Sg has neither an infimum nor a supremum. 
Thus Q does not satisfy (LUB) or (GLB). On the other hand, we certainly believe 
that there is a real number whose square is 2. But...why do we believe this? As we 
have seen, the existence of a real square root of every non-negative real number is a 
consequence of the Intermediate Value Theorem...which is of course a theorem that 
we have exalted but not yet proved. A more fundamental answer is that we believe 
that /2 exists in R because of the Dedekind completeness of R, i.e., according to 
Theorem 6.1 every nonempty bounded above subset of R has a supremum, so in 


particular Sp has a supremum, which must be V2. 


An interesting feature of Example 6.1 is that we can see that inf Sp exists, even 
though we have not as yet addressed the issue of whether R satisfies (GLB). In 
general, inf Si exists iff there is an element y < 0 in F with y? = 2. But okay: if 
in F we have a positive element x with 2? = 2, we necessarily must also have a 
negative element y with y? = 2: namely, y = —2. 

This turns out to be a very general phenomenon. 


THEOREM 6.2. Let F be an ordered field. 
a) Then F satisfies (LUB) iff it satisfies (GLB). 
b) In particular R satisfies both (LUB) and (GLB) and is (up to isomor- 
phism) the only ordered field with this property. 


Proor. a) I know two ways of showing that (LUB) <= (GLB). Both of 
these arguments is nice in its own way, and I don’t want to have to choose between 
them. So I will show you both, in the following way: I will use the first argument to 
show that (LUB) = > (GLB) and the second argument to show that (GLB) ==> 
(LUB). (In Exercise 6.1 below, you are asked to do things the other way around.) 
(LUB) = > (GLB): Let S C F be nonempty and bounded below by m. Consider 


—S={-a|xe€S}. 


Then —S' is nonempty and bounded above by —m. By (LUB), it has a least upper 
bound sup(—S'). We claim that in fact — sup(—S’)) is a greatest lower bound for S, 
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or more symbolically: 
inf S = —sup —S. 
You are asked to check this in Exercise 6.1 below. 
(GLB) = > (LUB): Let S be nonempty and bounded above by M. Consider 
U(S) = {x € F | x is an upper bound for S.}. 


Then U/(S) is nonempty: indeed M € U(S). Also U(S) is bounded below: indeed 
any s € S (there is at least one such s, since S # @!) is a lower bound for U/(S). By 
(GLB) U(S) has a greatest lower bound infU(S). We claim that in fact inf i/(S) is 
a least upper bound for S, or more succinctly, 


sup S = inf U(S). 
Once again, Exercise 6.1 asks you to check this. 
b) By Theorem 6.1a), R satisfies (LUB), and thus by part a) it satisfies (GLB). By 
Theorem 6.1b) R is the only ordered field satisfying (LUB), so certainly it is the 
only ordered field satifying (LUB) and (GLB). 


EXERCISE 6.1. a) Fill in the details of the proof of Theorem 6.2a). 
b) Let F be an ordered field, and let S be a subset of F'. Suppose that inf S 
exists. Show that sup —S' exists and 


sup —S = —inf S. 


c) Use part b) to give a second proof that (GLB) = > (LUB). 
d) Let F' be an ordered field, and let S' be a subset of F'. Define 


L(S) = {x € F | x is a lower bound for S.}. 
Suppose that sup £L(S) exists. Show that inf S exists and 
inf S = sup £(S). 
e) Use part d) to give a second proof that (LUB) => (GLB). 


The technique which was used to prove (LUB) ==> (GLB) is very familiar: we 
multiply everything in sight by —1. It seems likely that by now we have used this 
type of argument more than any other single trick or technique. When this has 
come up we have usually used the phrase “and similarly one can show...” Perhaps 
this stalwart ally deserves better. Henceforth, when we wish to multiply by —1 to 
convert < to >, max to min, sup to inf and so forth, we will say by reflection. 
This seems more appealing and also more specific than “similarly...”! 


In view of Theorem 6.2 it is reasonable to use the term Dedekind complete- 
ness to refer to either or both of (LUB), (GLB), and we shall do so. 


THEOREM 6.3. A Dedekind complete ordered field is Archimedean. 


PROOF. We will prove the contrapositive: let F' be a non-Archimedean ordered 
field: thus there exists x € F such that n < x for all n € Z*. Then the subset Z* 
of F is bounded above by z, so in particular it is nonempty and bounded above. 
So, if F were Dedekind complete then sup Z* would exist. 

But we claim that in no ordered field F does Z*+ have a supremum. Indeed, 
suppose that M = supZ*. It follows that for all n € Zt, n < M. But then it 
is equally true that for all n € Zt, n +1 < M, or equivalently, for all n € Z*, 
n < M—1,so M—1isasmaller upper bound for Z* than sup Zt: contradiction! 
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1.2. Calisthenics With Sup and Inf. 
The material and presentation of this section is partly based on [A, §1.3.13]. 


CONVENTION: Whenever sup S appears in the conclusion of a result, the state- 
ment should be understood as including the assertion that sup S exists, i.e., that 
S is nonempty and bounded above. Similarly for inf S: when it appears in the 
conclusion of a result then an implicit part of the conclusion is the assertion that 
inf S' exists, ie., that S is nonempty and bounded below.” 


PROPOSITION 6.4. Let S be a nonempty subset of R. 


a) Suppose S is bounded above. Then for every € > 0, there exists x € S 
such that supS —e <a <supS. 

b) Conversely, suppose M € R is an upper bound for S such that for all 
€>0, there exists x € S with M-—e<a<M. Then M =supS. 

c) Suppose S' is bounded below. Then for every € > 0, there exists x € S' such 
that inf S<a2< inf S+e. 

d) Conversely, suppose m € R is a lower bound for S such that for alle > 0, 
there exists x € S withm<a<mt+e. Thenm= inf S. 


PROOF. a) Fix e > 0. Since sup S is the least upper bound of S$ and sup S—e < 
sup S, there exists y € S with sup S —e < y. It follows that 


sup S —€ < min(y,sup S) < supS, 


so we may take x = min(y, sup S). 
b) By hypothesis, M is an upper bound for S and nothing smaller than M is an 
upper bound for S$, so indeed M = sup S. 

c),d) These follow from parts a) and b) by reflection. 


EXERCISE 6.2. Let a,b € R. Suppose that for alle > 0,a<b+e. Show: a< b. 
PROPOSITION 6.5. Let X,Y be nonempty subsets of R, and define 
X+Y={at+y|xeXx, ye Yh. 
a) Suppose X and Y are bounded above. Then 
sup(X + Y) =sup X +supY. 
b) Suppose X and Y are bounded below. Then 
inf(X + Y) = inf X + inf Y. 
PRooF. a) Let x € X,y € Y. Then x < supX and x < supY, sor+y < 


sup X +sup Y, and thus sup(X+Y) < sup X+sup Y. Now fix e > 0. By Proposition 
6.4 there are x € X andy € Y with sup X — 5 <a, supY — 5 <y, so 


sup X +supY <a+y+te. 


Since this folds for all « > 0, by Exercise 1.3 sup X +supY < sup(X + Y). 
b) This follows from part a) by reflection. 


Let X,Y be subsets of R. We write X < Y if forallae X andallyc Y,xr<y. 
(In a similar way we define X < Y,X >Y, X >Y.) 


2Notice that a similar convention governs the use of lim;-+- f(x), so this is nothing new. 
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EXERCISE 6.3. Let X,Y be subsets of R. Give necessary and sufficient condi- 
tions for X <Y and Y < X both to hold. (Hint: in the case in which X and Y 
are both nonempty, X =Y is necessary but not sufficient!) 


PROPOSITION 6.6. Let X,Y be nonempty subsets of R with X <Y. Then 
sup X < inf Y. 
PROOF. Seeking a contradiction, we suppose that inf Y < sup X. Put 
sup X — inf Y 
€ = ——_—_—_.. 
2 


By Proposition 6.4 there are x € X, y € Y with supX —e<avandy <infY +e. 
Since X < Y this gives 


supX —e<u<y<infY+e 


and thus 
sup X — inf Y < 2e =sup X — inf Y, 


a contradiction. 


PROPOSITION 6.7. Let X,Y be nonempty subsets of R with X CY. Then: 


a) If Y is bounded above, then sup X <supY. 
b) If Y is bounded below, then inf Y < inf X. 


EXERCISE 6.4. Prove Proposition 6.7. 
1.3. The Extended Real Numbers. 


As exciting and useful as this whole business with sup and inf is, there is one 
slightly annoying point: sup S and inf S are not defined for every subset of R. 
Rather, for sup S to be defined, S must be nonempty and bounded above, and for 
inf S' to be defined, S must be nonempty and bounded below. 

Is there some way around this? There is. It involves bending the rules a bit, 
but in a very natural and useful way. Consider the subset N of R. It is not bounded 
above, so it does not have a least upper bound in R. Because N contains arbitrar- 
ily large elements of R, it is not completely unreasonable to say that its elements 
approach infinity and thus to set sup N = +00. In other words, we are suggesting 
the following definition: 


e If S CR is unbounded above, then we will say sup S = +00. 
Surely we also want to make the following definition (“by reflection”!): 


e If S CR is unbounded below, then we will say inf S = —oo. 


These definitions come with a warning: -+too are not real numbers! They 
are just symbols suggestively standing for a certain type of behavior of a subset of 
R, in a similar (but, in fact, simpler) way as when we write lim,_,. f(a) = oo and 
mean that the function has a certain type of behavior near the point c. 


To give a name to what we have done, we define the extended real numbers 
[—co, co] = RU {+00} to be the real numbers together with these two formal 
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symbols —oo and oo. This extension is primarily order-theoretic: that is, we may 
extend the < relation to the extended real numbers in the obvious way: 


Va € R,-co <4 < oo. 


Conversely much of the point of the extended real numbers is to give the real 
numbers, as an ordered set, the pleasant properties of a closed, bounded interval 
[a, 6]: namely we have a largest and smallest element. 

The extended real numbers [—o0, oo] are not a field. In fact, we cannot even 
define the operations of + and - unrestrictedly on them. However, it is useful to 
define some of these operations: 


Vac ER, -o+x2=-co, r+0o=o@. 

Va € (0,00), ©-0O =, &-(—00) = —00. 
Va € (—00,0), v-00 = —00, &- (-—00) =~. 
00+ 00 = 00, 00: (—00) = —00, (—0o) - (—00) = ov. 
aria: 

oo =——00 


None of these definitions are really surprising, are they? If you think about it, 
they correspond to facts you have learned about manipulating infinite limits, e.g. 
if lim,_,. f(@) = co and lim,-_,. g(x) = 17, then lim,_,. f(x) + g(a) = co. However, 
certain other operations with the extended real numbers are not defined, for similar 
reasons. In particular we do not define 


coO- &, 


Why not? Well, again we might think in terms of associated limits. The above are 
indeterminate forms: if I tell you that lim,_,. f(a) = oo and lim,_,. g(x) = —on, 
then what can you tell me about limz-,. f(a) + g(x)? Answer: nothing, unless you 
know what specific functions f and g are. As a simple example, suppose 

5 + 2011, g(x) = 


1 i 
Te) 8) Ca 
Then lim,_,. f(@) = 00, limz-,. g(a) = —o0, but 
lim f(x) + g(x) = lim 2011 = 2011. 
Lc @T—c 


So oo — co cannot have a universal definition independent of the chosen func- 
tions. In a similar way, when evaluating limits 0 - oo is an indeterminate form: if 
limz+- f(z) = 0 and lim,_,. g(x) = 00, then lim,_,. f(x)g(x) depends on how fast 
f approaches zero compared to how fast g approaches infinity. Again, consider 
something like f(x) = (x — c)?, g(x) = eaeyt And similarly for &. 

These are good reasons. However, there are also more purely algebraic reasons: 
there is no way to define the above expressions in such a way to make the field 
axioms work out. For instance, let a € R. Then a+ co = ow. If therefore we were 
allowed to substract co from oo we would deduce a = co — ov, and thus oo — co 
could be any real number: that’s not a well-defined operation. 


Above we have alluded to the existence of ordered fields F' that do not satisfy 
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the Archimedean axiom, i.e., for which there exist elements x such that x > n for 
all n € Z*. In speaking about elements like x we sometimes call them infinitely 
large. This is a totally different use of “infinity” than the extended real numbers 
above. Indeed, no ordered field F' can have a largest element x, because it follows 
easily from the field axioms that for any x € F, x+1> a. The moral: although we 
call +00 “extended real numbers”, one should not think of them as being elements 
of a number system at all, but rather limiting cases of such things. 


One of the merits of this extended definition of sup S and inf S is that it works 
nicely with calculations: in particular, all of the “calisthenics” of the previous sec- 
tion have nice analogues for unbounded sets. We leave it to the reader to investigate 
this phenomenon on her own. In particular though, let’s look back at Proposition 
6.7: it says that, under conditions ensuring that the sets are nonempty and bounded 
above / below, that if X C Y CR, then 


sup X < sup Y, 
inf Y < inf X. 


This definition could have motivated our definition of sup and inf for unbounded 
sets, as follows: for n € Z and X C R, put 


X"™={xEX |a<n}, X,={ex eX |x>n}. 
The idea here is that in defining X” we are cutting it off at n in order to force it 
to be bounded above, but in increasingly generous ways. We have 
b uae Guraneen ae: 4 


and also 
be et 
n=0 


in other words, every element of X is a subset of X” for some n (this is precisely the 
Archimedean property). Applying Proposition 6.7, we get that for every nonempty 
subset X of R, 


sup X° < sup X! < sup X27 <...supX"<.... 
Suppose moreover that X is bounded above. Then some N € Z* is an upper 
bound for X, ie, X = XN = XN+1 — ..., so the sequence sup X” is eventually 
constant, and in particular lim,...sup X” = sup X. On the other hand, if X 


is bounded above, then the sequence sup X” is not eventually constant; in fact it 
takes increasingly large values, and thus 


lim sup X” = oo. 
noo 


Thus if we take as our definition for sup X, lim; sup X”, then for X which is 
unbounded above, we get sup X = lim,_,.. sup X” = oo. By reflection, a similar 
discussion holds for inf X. 


There is, however, one last piece of business to attend to: we said we wanted 
sup S' and inf S to be defined for all subsets of R: what if S = @? There is an 
answer for this as well, but many people find it confusing and counterintuitive at 
first, so let me approach it again using Proposition 6.7. For each n € Z, consider 
the set P, = {n}: ie., P, has a single element, the integer n. Certainly then 
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inf P, = sup P, =n. So what? Well, I claim we can use these sets P, along with 
Proposition 6.7 to see what inf @ and sup @ should be. Namely, to define these 
quantities in such a way as to obey Proposition 6.7, then for all n € Z, because 
@ Cc {n}, we must have 

sup Z < sup{n} =n 
and 

inf @ > inf{n} =n. 
There is exactly one extended real number which is less than or equal to every 
integer: —oo. Similarly, there is exactly one extended real number which is greater 
than or equal to every integer: oo. Therefore the inexorable conclusion is 


sup S = —oo, inf 9 = oo. 


Other reasonable thought leads to this conclusion: for instance, in class I had a lot 
of success with the “pushing” conception of suprema and infima. Namely, if your 
set S is bounded above, then you start out to the right of every element of your 
set — i.e., at some upper bound of S — and keep pushing to the left until you can’t 
push any farther without passing by some element of S. What happens if you try 
this with @? Well, every real number is an upper bound for ©, so start anywhere 
and push to the left: you can keep pushing as far as you want, because you will 
never hit an element of the set. Thus you can push all the way to —oo, so to speak. 
Similarly for infima, by reflection. 


2. Intervals and the Intermediate Value Theorem 


2.1. Convex subsets of R. 


We say that a subset S of R is convex if for all x < y € S, the entire interval 
[x,y] lies in S. In other words, a convex set is one that whenever two points are in 
it, all in between points are also in it. 


EXAMPLE 6.2. The empty set @ is convex. For any x € R, the singleton set 
{x} is convex. In both cases the definition applies vacuously: until we have two 
distinct points of S, there is nothing to check! 


EXAMPLE 6.3. We claim any interval is conver. This is immediate — or it 
would be, if we didn’t have so many different kinds of intervals to write down and 
check. One must check that the definition applies to intervals of the following forms: 


(a,b), [a,b), (a,b], [a,b], (—00, 6), (—0o, b], (a, 00), [a, 00), (—00, 00). 
All these verifications are routine appeals to things like the transitivity of < and >. 
Are there any nonempty convex sets other than intervals? (Just to be sure, we count 
{x} = [a, 2] as an interval.*) A little thought suggests that the answer should be no. 
But more thought shows that if so we had better use the Dedekind completeness of 


R, because if we work over Q with all of the corresponding definitions then there 
are nonempty convex sets which are not intervals, e.g. 


S={xEQ|a* <2}. 


3However, we do not wish to say whether the empty set is an interval. Throughout these 
notes the reader may notice minor linguistic contortions to ensure that this issue never arises. 
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This has a familiar theme: replacing Q by R we would get an interval, namely 
(-V2, /2), but once again +2 ¢ Q. When one looks carefully at the definitions 
it is no trouble to check that working solely in the rational numbers S is a convex 
set but is not an interval. 


Perhaps the above example seems legalistic, or maybe even a little silly. It re- 
ally isn’t: one may surmise that contemplation of such examples led Dedekind 
to his construction of the real numbers via Dedekind cuts. This construction 
may be discussed at the end of this course. Most contemporary analysts prefer 
a rival construction of R due to Cauchy using Cauchy sequences. I agree that 
Cauchy’s construction is simpler. However, both are important in later mathemat- 
ics: Cauchy’s construction works in the context of a general metric space (and, 
with certain modifications, in a general uniform space) to construct an associ- 
ated complete space. Dedekind’s construction works in the context of a general 
linearly ordered set to construct an associated Dedekind-complete ordered set. 


THEOREM 6.8. Every nonempty convex subset D of R is an interval. 


PRooFr. We are given a nonempty convex subset D of R and we want to show 
it is an interval, but as above an interval can have any one of nine basic shapes. It 
may be quite tedious to argue that one of nine things must occur! 

So we just need to set things up a bit carefully: here goes: let a € [—00, co) be 
the infimum of D, and let b € (—oo, 00] be the supremum of D. Let I = (a,b), and 
let I be the closure of J, ie., if a is finite, we include a; if b is finite, we include b. 
Step 1: We claim that JC D CT. Let # € I. 

Case 1: Suppose I = (a,b) with a,b € R. Let z € (a,b). Then, since z >a = 
inf D, there exists c € D with c < z. Similarly, since z < b = sup D, then there 
exists d € D with z < d. Since D is convex, z € D. Now suppose z € D. We must 
have inf D=a<z<b=supD. 

Case 2: Suppose I = (—oo,b), and let z € I. Since D is unbounded below, 
there exists a € D with a < z. Moreover, since z < sup D, there exists b € D such 
that z < b. Since D is convex, z € D. Next, let z € D. We wish to show that 
zEel= (—oo, b]; in other words, we want z < b. But since z € D and b = sup D, 
this is immediate. Thus JC D CT. 

Case 3: Suppose I = (a, 00). This is similar to Case 2 and is left to the reader. 

Case 4: Suppose I = (—oo,00) = R. Let z € R. Since D is unbounded below, 
there exists a € D with a < z, and since D is unbounded above there exists b € D 
with z <b. Since D is convex, z € D. Thus J = D=T=R. 
Step 2: We claim that any subset D which contains J and is contained in J is an 
interval. Indeed I and I are both intervals, and the only case in which there is any 
subset D strictly in between them is I = (a,b) with a,b € R — in this case D could 
also be [a,b) or (a,b], and both are intervals. 


EXERCISE 6.5. (R. Freiwald): Let F' be an ordered field. For a subset S of F, 
we define the associated downset 


S,={xeF|xa<-s for somes ce S}. 


a) Show thatS#@ => SL AO. 
b) Show that S is bounded above <= > S\ is bounded above. 
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c) Show that S has a supremum in F <=> S| has a supremum in F, and 
if so then we have sup S = sup S$. 

d) In any ordered field F', we say a subset S of F is convex if x,y € S with 
x<y, then for all z witha <z<y,ze€S. Show that for all nonempty 
subsets S of F’, the downset S\ 1s convex. 

e) Deduce the converse to Theorem 6.8: if F is an ordered field in which every 
nonempty convex subset is an interval, then F' is Dedekind complete. 


Recall that a function f : D > R satisfies the Intermediate Value Property 
(IVP) if for all [a,b] C D, for all L in between f(a) and f(b) is of the form f(c) 
for some c € (a,b). As you may well have noticed, the IVP is closely related to the 
notion of a convex subset. The following result clarifies this connection. 


THEOREM 6.9. For f: DCR-R, the following are equivalent: 


(i) For all [a,b] C D, the subset f(a, b]) is conver. 
(ii) The function f satisfies the Intermediate Value Property. 
(iii) For any interval IC D, the set f(I) is an interval. 


PROOF. (i) => (ii): For all [a,b] c D, f({a, b]) is a convex subset containing 
f(a) and f(b), hence it contains all numbers in between f(a) and f(b). 
(ii) => (iii): Suppose that f satisfies IVP, and let I C D be an interval. We want 
to show that f(Z) is an interval. By Theorem 8.1 it suffices to show that f(J) is 
convex. Assume not: then there exists a < b € I and some L in between f(a) and 
f(b) such that L A f(c) for any c € I. In particular L ¥ f(c) for any c € [a, I, 
contradicting the Intermediate Value Property. 
(iii) — > (i): This is immediate: [a,}] is an interval, so by assumption f([a, 6]) is 
an interval, hence a convex subset. 


2.2. The Strong Intermediate Value Theorem. 


THEOREM 6.10. (Strong Intermediate Value Theorem) If f : I > R is contin- 
uous, then f satisfies the Intermediate Value Property and thus f(I) is an interval. 


PROOF. Step 1: We make the following CLAIM: if f : [a,b] > R is continuous, 
f(a) < 0 and f(b) > 0, then there exists c € (a,b) such that f(c) =0. 
PROOF OF CLAIM: Let S = {a € [a,b] | f(a) < 0}. Since a € S, S' is nonempty. 
Moreover S is bounded above by 6. Therefore S has a least upper bound c = sup S. 
It is easy to see that we must have f(c) = 0. Indeed, if f(c) < 0, then — as we have 
seen several times — there exists 6 > 0 such that f(x) < 0 for all x € (c—6,c+9), 
and thus there are elements of S larger than c, contradicting c = sup S. Similarly, 
if f(c) > 0, then there exists 6 > 0 such that f(x) > 0 for all  € (c—4,c+9), in 
which case any element of (c — 6,c) gives a smaller upper bound for S than c. By 
the process of elimination we must have f(c) = 0! 
Step 2: We will show that f satisfies the Intermediate Value Property: for all 
[a,b] C I, and any L in between f(a) and f(b), we must find c € (a,b) such that 
f(c) = L. If f(a) = f(b) there is nothing to show. If f(a) > f(b), then we may 
replace f by —f (this is still a continuous function), so it is enough to treat the case 
f(a) < L < f(6). Now consider the function g(x) = f(x)—L. Since f is continuous, 
so is g; moreover g(a) = f(a) — L < 0 and g(b) = f(b) — L > 0. Therefore by Step 
1 there exists c € (a,b) such that 0 = g(c) = f(c) — L, ie., such that f(c) = L. 
Step 3: By Step 2 and Theorem 6.9, f(J) is an interval. 
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Theorem 6.10 is in fact a mild improvement of the Intermediate Value Theorem we 
stated earlier in these notes. This version of IVT applies to continuous functions 
with domain any interval, not just an interval of the form [a,b], and includes a 
result that we previously called the Interval Image Theorem. 


2.3. The Intermediate Value Theorem Implies Dedekind Complete- 
ness. 


THEOREM 6.11. Let F' be an ordered field such that every continuous function 
f:F-F satisfies the Intermediate Value Property. Then F is Dedekind complete. 


PROOF. We will prove the contrapositive: suppose F’ is not Dedekind complete, 
and let S Cc F be nonempty and bounded above but without a least upper bound 
in F. Let U(S) be the set of upper bounds of S. We define f : F > F by: 

e f(x) =-1, ifx gU(S), 

e f(z) =1, 2 €U(S). 

Then f is continuous on F' — indeed, a point of discontinuity would occur only at 
the least upper bound of S, which is assumed not to exist. Moreover f takes the 
value —1 — at any element s € S, which cannot be an upper bound for S because 
then it would be the maximum element of S — and the value 1 at any upper bound 
for S (we have assumed that S is bounded above so such elements exist), but it 
never takes the value zero, so f does not satisfy IVP. 


EXERCISE 6.6. Show in detail that the function f : F + F constructed in the 
proof of Theorem 6.11 is continuous at every element of F’. 


3. The Monotone Jump Theorem 
THEOREM 6.12. (Monotone Jump) Let f : I > R be weakly increasing. 


a) Letc be an interior point of I. Then lim,_,.- f(x) and lim,_,,+ f(x) exist, 
and 


lim f(«) < f(c) < lim f(a). 
@—c7 a—ct 
b) Suppose I has a left endpoint a. Then lim,_,,+ f(x) exists and is at least 
f(a). 
c) Suppose I has a right endpoint b. Then lim,,,- f(x) exists and is at most 
f(e). 
PROOF. a) Step 0: As usual, we may f is weakly increasing. We define 
L=({f(x)|vel,«c<c}, R={f(x)|xeel,¢> ch}. 
Since f is weakly increasing, L is bounded above by f(c) and U is bounded below 
by f(c). Therefore we may define 
(=sup L,r = inf R. 
Step 1: For all x <c, f(x) < f(c), f(c) is an upper bound for L, sol < f(c). For 
alle < a, f(c) < f(x), so f(c) is a lower bound for R, so f(c) < +. Thus 
(23) I< fle) <t 
Step 2: We claim lim,_,.- f(a) =. To see this, let « > 0. Since I is the least upper 


bound of Z and [—e < I, [—e is not an upper bound for L: there exists xp < ¢ 
such that f(xo) > t—. Since f is weakly increasing, for all 7p < x < c we have 


[—e< f(to) < f(a) << I+e. 
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Thus we may take 6 = c— 20. 
Step 3: We claim lim,_,.4 f(a) =v: this is shown as above and is left to the reader. 
Step 4: Substituting the results of Steps 2 and 3 into (23) gives the desired result. 
b) and c): The arguments at an endpoint are routine modifications of those of part 
a) above and are left to the reader as an opportunity to check her understanding. 


THEOREM 6.13. Let f : I — R be monotone. The following are equivalent: 


(i) The subset f(I) is an interval. 
(ii) The function f is continuous. 


Proor. As usual, it is no loss of generality to assume f is weakly increasing. 
(i) = > (ii): If f is not continuous on all of J, then by the Monotone Jump 
Theorem f(J) fails to be convex. In more detail: suppose f is discontinuous at c. 
If c is an interior point then either lim,_,.- f(x) < f(c) or f(c) < limg4.+ f(z). 
In the former case, choose any b € I, b < c. Then f(Z) contains f(b) < f(c) but 
not the in-between point lim,_,.- f(x). In the latter case, choose any d € I, c < d. 
Then f (J) contains f(c) < f(d) but not the in-between point lim,_,.+ f(a). Similar 
arguments hold if c is the left or right endpoint of J: these are left to the reader. 
Thus in all cases f() is not convex hence is not an interval. 
(ii) ==> (i): This follows immediately from Theorem 6.10. 


With Theorems 6.10 and 6.13 in hand, we get an especially snappy proof of the 
Continuous Inverse Function Theorem. Let f : J + R be continuous and injective. 
By Theorem 6.10, f(Z) = J is an interval. Moreover f : I > J is a bijection, with 
inverse function f~! : J 4 I. Since f is monotone, so is f~+. Moreover f~!(J) = I 
is an interval, so by Theorem 6.13, f~! is continuous! 


4. Real Induction 


THEOREM 6.14. (Principle of Real Induction) Let a < b be real numbers, let 
SC [a, 6], and suppose: 
(RI1) ae S, 
(RI2) for alla € S, ifx #b there exists y > x such that [x,y] CS. 
(RI3) For allx ER, if [a,x) € S, thenz eS. 
Then S = [a, bj. 


PRooF. Seeking a contradiction we suppose not: 5’ = [a,b] \ S is nonempty. 
It is bounded below by a, so has a (finite!) greatest lower bound inf S’. However: 
Case 1: inf S’ = a. Then by (RI1), a € S, so by (R12), there exists y > a such that 
[a,y] C S, and thus y is a greater lower bound for S’ then a = inf S$’: contradiction. 
Case 2: a < inf S’ € S. If inf S’ = b, then S = [a,b]. Otherwise, by (RI2) there 
exists y > inf S’ such that [inf S’, y’] C S, contradicting the definition of inf $’. 
Case 3: a < inf S’ € S’. Then [a, inf S’) C S, so by (RI8) inf S’ € S: contradiction! 


EXAMPLE 6.4. Let us reprove the Intermediate Value Theorem. The Interme- 
diate Value Theorem is equivalent to: let f : [a,b] + R be continuous and nowhere 
zero. If f(a) > 0, then f(b) > 0. We prove this by Real Induction. Let 

S = {x € [a,b] | f(x) > Of. 


Then f(b) > 0 iffb eS. We will show S = [a,b] by real induction, which suffices. 
(RI1) By hypothesis, f(a) > 0, soae S. 
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(RI2) Letx Ee S, x <b, so f(x) >0. Since f is continuous at x, there exists 6 > 0 
such that f is positive on [x,x +6], and thus [x,x +06] CS. 

(RI3) Let x € (a,b] be such that [a,x) C S, t.e., f is positive on [a,x). We claim 
that f(x) > 0. Indeed, since f(x) 4 0, the only other possibility is f(x) <0, but if 
so, then by continuity there would exist 6 > 0 such that f is negative on [x — 6, a], 
i.e., f is both positive and negative at each point of |x — 6,x]: contradiction! 


The following result shows that Real Induction not only makes use of the Dedekind 
completeness of R but actually carries the full force of it. 


THEOREM 6.15. In an ordered field F’, the following are equivalent: 
(i) F is Dedekind complete: every nonempty bounded above subset has a supremum. 
(ii) F satisfies the Principle of Real Induction: for alla <b € F,, a subset SC [a,b] 
satisfying (RI1) through (RI3) above must be all of [a, b]. 


PRooF. (i) => (ii): This is simply a restatement of Theorem 6.14. 
(ii) => (i): Let T C F be nonempty and bounded below by a € F. We will show 
that T has an infimum. For this, let S be the set of lower bounds m of T with 
a<m. Let b be any element of T. Then S C [a,b]. 
Step 1: Observe that be S <= > 6 = infT. In general the infimum could be 
smaller, so our strategy is not exactly to use real induction to prove S = |{a,)]. 
Nevertheless we claim that S satisfies (RIL) and (RI8). 
(RI1): Since a is a lower bound of T with a < a, we haveae€ S. 
(RI3): Suppose x € (a, b] and [a, x) C S, so every y € [a, x) is a lower bound for T. 
Then x is a lower bound for T: if not, there exists t € T such that t < x; taking 
any y € (t,x), we get that y is not a lower bound for T either, a contradiction. 
Step 2: Since F' satisfies the Principle of Real Induction, by Step 1 S = [a,b] iff S 
satisfies (RI2). If S = [a, 6], then the element b € is a lower bound for T, so it must 
be the infimum of T. Now suppose that S 4 [a,b], so by Step 1 S does not satisfy 
(RI2): there exists x € S, « < b such that for any y > a, there exists z € (x,y) such 
that z € S, i.e., z is not a lower bound for T. In other words x is a lower bound 
for T and no element larger than x is a lower bound for T...so « = inf T. 


Like Dedekind completeness, “Real Induction” depends only on the ordering re- 
lation < and not on the field operations + and -. In fact, given any ordered set 
(F, <) —i.e., we need not have operations + or - at all — it makes sense to speak of 
Dedekind completeness and also of whether an analogue of Real Induction holds. 
In [C111], I proved that Theorem 6.15 holds in this general context: an ordered set 
F is Dedekind complete iff the only it satisfies a “Principle of Ordered Induction.” 


5. The Extreme Value Theorem 


THEOREM 6.16 (Extreme Value Theorem). Let f : [a,b] + R be continuous. 


a) The function f is bounded. 
b) The function f attains a minimum and maximum value. 


PRoor. a) Let S = {x € [a,}] | f : [a,z] — R is bounded}. 
(RI1): Evidently a € S. 
(RI2): Suppose x € S, so that f is bounded on [a,z]. But then f is continuous 
at x, so is bounded near x: for instance, there exists 6 > 0 such that for all 
y € [c—6,x+], |f(y)| < |f(x)|+1. So f is bounded on [a, 2] and also on [z, x + 4] 
and thus on [a,x + d]. 
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(RI3): Suppose that x € (a, b] and [a,x) C S. Now beware: this does not say that 
f is bounded on [a, x): rather it says that for all a < y < x, f is bounded on [a, y]. 
These are really different statements: for instance, f(x) = —4, is bounded on {0, y] 
for all y < 2 but it is not bounded on [0,2). But, as usual, the key feature of this 
counterexample is a lack of continuity: this f is not continuous at 2. Having said 
this, it becomes clear that we can proceed almost exactly as we did above: since f 
is continuous at x, there exists 0 < 6 < #—a such that f is bounded on [x — 6, a]. 
But since a < x —6 < x we know also that f is bounded on [a,x — 6], so f is 
bounded on [a, «]. 


b) Let m = inf f({a, b]) and M = sup f([a,b]). By part a) we have 
-o<m<M<ow. 


We want to show that there exist tm,vu € [a,b] such that f(am) =m, f(e«m) = M, 
i.e., that the infimum and supremum are actually attained as values of f. Suppose 
that there does not exist x € [a,b] with f(~) =m: then f(x) > m for all « € [a, }] 
and the function gm : [a,b] > R by gm(x) = Feo is defined and continuous. By 
the result of part a), gm is bounded, but this is absurd: by definition of the infimum, 
f(a) —™m takes values less than 4 for any n € Z+ and thus g,, takes values greater 
than n for any n € Zt and is accordingly unbounded. So indeed there must exist 
Lm € [a,b] such that f(vm) =m. Similarly, assuming that f(x) < M for all x € 
[a, b] gives rise to an unbounded continuous function gu : [a,b] > R, x! 


1 
? M—Flay’ 


contradicting part a). So there exists yy € [a,b] with f(r) = M. 


6. The Heine-Borel Theorem 


Let S C R, and let {X;}ier be a family of subsets of R. We say that the family 
{X;} covers S if S C U;-,; Xi: in words, this simply means that every element 
x € S is also an element of X; for at least one 7. 


THEOREM 6.17. (Heine-Borel) Let {Ui}icr be any covering of the closed, 
bounded interval by open intervals U;. Then the covering {U;}icr has a finite 
subcovering: there is a finite subset J C I such that every x € [a,b] lies in U; for 
some j EJ. 


PRooF. For an open covering U = {U;}ier of [a, b], let 
S = {x € [a,b] | Un [a, 2] has a finite subcovering}. 


We prove S' = [a,b] by Real Induction. (RI1) is clear. (RI2): If Uj,...,Un covers 
[a, x], then some U; contains [x, 7+] for some 6 > 0. (RI3): if a,x) C S, let iz € I 
be such that x € U;,, and let 6 > 0 be such that [x — 6,2] € U;,. Since r—O € S, 
there is a finite J C I with U;., Ui D [a,x — 4], so {Ui}iey UU, covers [a, a]. 


EXERCISE 6.7. The formulation of the Heine-Borel Theorem given above is 
superficially weaker than the standard one. A subset of R is called open if it is 
a union of open intervals. In the usual statement of the Heine-Borel Theorem 
the U;’s are allowed to be arbitrary open subsets of R. Show that this apparently 
more general version can be deduced from Theorem 6.17. (Suggestion: for each 
x € [a,b], there is an open subset U, containing x. By definition of open subset, Uz 
must contain some open interval I, containing x. Apply Theorem 6.17 to the open 
covering {I,,}.) 
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7. Uniform Continuity 


7.1. The Definition; Key Examples. Let J be an interval and f: J > R. 
Then f is uniformly continuous on I if for every « > 0, there exists a d > 0 such 
that for all 71, x2 € J, if jay — 2| < 6 then |f(x1) — f(xa)| <e. 


In order to show what the difference is between uniform continuity on J and “mere” 
continuity on J — i.e., continuity at every point of J — let us rephrase the standard 
e-6 definition of continuity using the notation above. Namely: 


A function f : J > R is continuous on I if for every « > 0 and every x; € I, there 
exists 6 > 0 such that for all x2 € I, if |x; — x2| < 6 then |f(#1) — f(x2)| <e. 


These two definitions are eerily (and let’s admit it: confusingly, at first) similar: 
they use all the same words and symbols. The only difference is in the ordering 
of the quantifiers: in the definition of continuity, player two gets to hear the value 
of € and also the value of x; before choosing her value of 6. In the definition of 
uniform continuity, player two only gets to hear the value of €: thus, her choice of 6 
must work simultaneously — or, in the lingo of this subject, uniformly — across all 
values of x, € I. That’s the only difference. Of course, switching the order of quan- 
tifiers in general makes a big diffference in the meaning and truth of mathematical 
statements, and this is no exception. Let’s look at some simple examples. 


EXAMPLE 6.5. Let f: RR by f(z) =max+b, m0. We claim that f is 
uniformly continuous on R. In fact the argument that we gave for continuity 
long ago shows this, because for every € > 0 we took 6 = ia Although we used this 
6 to show that f is continuous at some arbitrary point c € R, evidently the choice 
of 6 does not depend on the point c: it works uniformly across all values of c. Thus 


f is uniformly continuous on R. 


EXAMPLE 6.6. Let f : RR by f(x) =2?. This time I claim that our usual 
proof did not show uniform continuity. Let’s see it in action. To show that f is 
continuous at c, we factored x2 — c into (x —c)(a +c) and saw that to get some 
control on the other factor x +c we needed to restrict x to some bounded interval 
around c, say [c—1,c+1]. On this interval |jx+c| < |z|+|c] < |e|+1+]e] < 2|e/+1. 
So by taking 6 = min(1, erat) we found that if |x — cl <6 then 


If(2) — (| = |e — ella 4 


€ 
cel < -(Qlc/ +1) =e. 

But the above choice of 6 depends on c. So it doesn’t show that f is uniformly 
continuous on R. In fact the function f(x) = x? is not uniformly continuous on 
R. For instance, take ¢ = 1. If it were uniformly continuous, there would have to 
be some 6 > 0 such that for all 71,22 € R with |v, — x9| < 6, |x} — 23| < €. But 
this is not possible: take any 6 > 0. Then for anyx € R, x andx+ f are less than 
6 apart, and |x? — (x + $)?| = |xd + a: But if I get to choose x after you choose 
6, this expression can be made arbitrarily large. In particular, if x = +, then it is 
strictly greater than 1. So f is not uniformly continuous on R. 


EXERCISE 6.8. Let f : R > R be a polynomial. Show: f is uniformly continuous 
if and only if f is linear. 
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So that’s sad: uniform continuity is apparently quite rare. But wait! What if the 
domain is a closed, bounded interval I? For instance, by restricting f(x) = 2? 
to any such interval, it 7s uniformly continuous. Indeed, we may as well assume 
I = [—M, M], because any I is contained in such an interval, and uniform conti- 
nuity on [—M, M] implies uniform continuity on I. Now we need only use the fact 
that we are assuming |c| < M to remove the dependence of 6 on c: since |c| < M 
we have Wel) > oa so for € > 0 we may take 6 = min(1 ). This shows 
that f(x) = 2? is uniformly continuous on [—M, M]. 


eae ae 
> 2M+1 


It turns out that one can always recover uniform continuity from continuity by 
restricting to a closed bounded interval: this is the last of our Interval Theorems. 


7.2. The Uniform Continuity Theorem. 


Let f: J > R. For €,6 > 0, let us say that f is (€,6)-UC on J if for all 21,72 € J, 
|v —X2|<d => |f(xi) — f(e2)| < €. This is a sort of halfway unpacking of the 
definition of uniform continuity. More precisely, f : J + R is uniformly continuous 
iff for all « > 0, there exists 6 > 0 such that f is (€,6)-UC on I. 


The following small technical argument will be applied twice in the proof of the 
Uniform Continuity Theorem, so advance treatment of this argument should make 
the proof of the Uniform Continuity Theorem more palatable. 


LEMMA 6.18 (Covering Lemma). Let a <b <c< d be real numbers, and let 
f :[a,d] > R. Suppose that for real numbers €1,61, 62 > 0, 
e f is (€,d,)-UC on [a,c] and 
e f is (€,d2)-UC on [b, dj. 
Then f is (€,min(d1,52,c — b))-UC on [a, BI. 


PROOF. Suppose x1 < 22 € J are such that |x, — x2| < 6. Then it cannot be 
the case that both x, < b and c < xo: if so, v2 — x, > c—b > 6. Thus we must 
have either that b < 41 < @ or 41 <a <c. If b < a < x, then 1,22 € [b,d 
and |%1 — x2| < 6 < da, so |f(a1) — f(x2)| < €. Similarly, if 71 < x2 < c¢, then 
£1, 2 € [a,c] and |x; — x2] < 6 < 64, so | f(x1) — f(xa)| <e. 


THEOREM 6.19 (Uniform Continuity Theorem). Let f : [a,b] + R be continu- 
ous. Then f is uniformly continuous on [a, b]. 


PRoor. For € > 0, let S(€) be the set of x € [a, b] such that there is 6 > 0 such 
that f is (€,d)-UC on [a,z]. To show that f is uniformly continuous on [a, )], it 
suffices to show that S'(e) = [a, b] for all « > 0. We will show this by Real Induction. 
(RIL): Trivially a € S(e): f is (€,5)-UC on [a, a] for all 6 > 0! 

(RI2): Suppose 2 € S(e), so there exists 6; > 0 such that f is (€,6,)-UC on 
[a,x]. Moreover, since f is continuous at x, there exists d2 > 0 such that for all 
c € [x, +a], |f(e)—f(a)| < §. Why §? Because then for all cy, cz € [v—d2, x +g], 
[F(e1) — Flee) = [Fler) — Fa) + Fe) — Flea)| < [fle.) — F@)| + IF (e2) — F@)| < €. 
In other words, f is (€, d2)-UC on [x — 62,2 +62]. We apply the Covering Lemma to 
f with a < x — 62 <x < 2+. to conclude that f is (€,min(d, 02,7 — (# — 42))) = 
(e, min(d1, 62))-UC on [a, x + 42]. It follows that [#, x + d2] C S(e). 

(RI3): Suppose [a,z) C S(e). As above, since f is continuous at x, there exists 


122 6. COMPLETENESS 


6; > 0 such that f is (€,6,)-UC on [a — 61, a]. Since « — 4 < x, by hypothesis there 
exists J2 such that f is (€, d2)-UC on [a, x — a), We apply the Covering Lemma to f 
with a < x—6, < x- a < x to conclude that f is (€, min(61, do, 7— Sy (w—61))) = 


2 
(ce, min(, 5g))-UC on [a,z]. Thus z € S(e). 


8. The Bolzano-Weierstrass Theorem For Subsets 


Let S C R. We say that x € R is a limit point of S if for every 6 > 0, there 
exists s € S with 0 < |s—a| < 6. Equivalently, x is a limit point of S if every open 
interval J containing x also contains an element s of S which is not equal to x. 


PROPOSITION 6.20. For S CR anda ER, the following are equivalent: 


(i) Every open interval I containing x also contains infinitely many points of 
S. 
(ii) The point x is a limit point of S. 


EXAMPLE 6.7. If S =R, then every x € R is a limit point. More generally, if 
S CR is dense — 1.e., if every nonempty open interval I contains an element of S 
— then every point of R is a limit point of S. In particular this holds when S = Q 
and when S=R\Q. Note that these examples show that a limit point x of S may 
or may not be an element of S: both cases can occur. 


EXAMPLE 6.8. If S CT and x is a limit point of S, x is a limit point of T. 
EXAMPLE 6.9. No finite subset S of R has a limit point. 


EXAMPLE 6.10. The infinite subset Z has no limit points: indeed, for any 
x ER, take I = (x—1,4+4+1). Then I is bounded so contains only finitely many 
integers. 


EXAMPLE 6.11. More generally, let S be a subset such that for all M > 0, 
SO[—-M,M] is finite. Then S has no limit points. 


THEOREM 6.21. (Bolzano- Weierstrass) 
Every infinite subset A of [a,b] has a limit point. 


Proor. Let AC [a, 6], and let S' be the set of x in [a,b] such that if AN [a, 2] 
is infinite, it has a limit point. It suffices to show S = [a,b], which we will do by 
Real Induction. (RI1) is clear. (RI2) Suppose z € [a,b) NS. If AN [a, x] is infinite, 
then it has a limit point and hence so does AN [a,b]: thus S = [a,b]. If for some 
5 >0, AN [a,x + 4] is finite, then [z,2 + 6] C S. Otherwise AN [a, 2] is finite but 
AN |[a, z+ 6] is infinite for all 6 > 0, and then z is a limit point for A and S = [a,b 
as above. (RI8) If [a,x) C S, then: either AN [a, y] is infinite for some y < z, so 
x € S; or AN|a, a] is finite, so x € S'; or AN[a, y] is finite for all y < x and AN[a, x 
is infinite, so x is a limit point of AN [a,z] and zé S. 


Later we will give a “sequential version” of Bolzano-Weierstrass and see that it is 
equivalent to Theorem 6.21 above. 


9. Tarski’s Fixed Point Theorem 


Mainly as a further showpiece for Real Induction, we give here a special case of 
a theorem of the great logician A. Tarski [Ta55]. It is a fixed point theorem: 
that is, a theorem which gives conditions on a function f : X — X for there to 
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be a point c € X with f(c) = c. Fixed point theorems are ubiquitously useful 
throughout mathematics, and further fixed point theorems for functions defined on 
subintervals of R will be given when we study infinite sequences later on. However 
Tarski’s theorem has a quite different flavor from the fixed point theorems to come 
in that the function f is not assumed to be continuous! 


THEOREM 6.22. (Tarski Fixed Point Theorem) Let f : [a,b] > [a,b] be a weakly 
increasing function. Then f has a fixed point: there is c € [a,b] with f(c) =c. 


PROOF. Seeking a contradiction we suppose f has no fixed point. Put 
S = {ax € [a,b] | f(x) > z}. 
We will show S = [a, b] by Real Induction. But then b € S, so f(b) > b, contradict- 
ing the fact that b is the largest element of [{a, b]! 
(RIL) Since a = minfa,b], we have f(a) > a hence (there is no fixed point!) 
f(a) >a. 
(RI2) Let x € [a,b) NS. Put y := f(x), sox <y. Forxz <z<y, since f is weakly 
increasing we have 
f(z) 2 f@)=y 22, 

and thus (again, no fixed point!) f(z) > z. Hence [x,y] C S. 
(RI3) Let x € (a, b], suppose [a,z) C S and put y = f(x). Ify <a then y € S so 

f(a) =y < fy), 


contradicting the fact that f is weakly increasing. 


EXERCISE 6.9. a) Give another (more standard) proof of Theorem 6.22 
by showing that sup{x € [a,b] | f(x) > x} ts a fixed point of f. 
b) Which argument do you prefer? 


EXERCISE 6.10. Find an application of Theorem 6.22 to calculus. 


I am indebted to J. Propp for alerting me to this application of Real Induction. 


CHAPTER 7 


Differential Miscellany 


1. L’H6pital’s Rule 


We have come to the calculus topic most hated by calculus instructors: L’H6pital’s 


Rule. This result gives a criterion for evaluating indeterminate forms 8 or =. 


The following succinct formulation is taken from [R, Thm. 5.13], as is the proof. 


THEOREM 7.1. Let -c0o <a<b< oo. Let f,g: (a,b) > R be differentiable. 
a) We suppose that 


and also that either 
(i) lim, .5- f(x) = lim, 5,- g(x) = 0, or 


(it) limg_.p- g(a) = too. 
Then lim,_,p- oo =A. 


b) The analogous statements with lim,.,- replaced everywhere by lim, .,+ hold. 


PROOF. a) Step 1: Suppose that A < oo, let a be any real number which is 
greater than A, and let 3 be such that A < 8 < a. We will show that there exists 


c € (a,b) such that for all x > c, a <a. 


First, since Ee — A < oo, there is c € (a,b) such that for all x > c, fe < B. 
Let c< x < y <b. By Cauchy’s Mean Value Theorem, there is ¢ € (x, y) such that 
f(xe)- fH) _ fF) 
g(z)-g(y) g(t) 

Suppose first that (i) holds. Then by letting « approach 6 in (24) we get fe < 
B <a for all c < y < b, which is what we wanted to show. 

Next suppose that (ii) holds. Fix y in (24) and choose c; € (y,b) such that c, < 
x <b implies g(x) > g(y) and g(x) > 0. Multiplying (24) by Ba gives 


PIDs 3 (2 = a) 


(24) <6 


J 


g(x) g(x) 
and then a little algebra yields 
f(x) gy) , FY) 
25 <B-£6 . 
— glo) g(x) * g(a) 
Letting x approach 6, we find: there is c € (c1,b) such that for all x > c, oS <a. 


Step 2: Suppose A > —oo. Then arguing in a very similar manner as in Step 1 
we may show that for any a < A there exists c € (a,b) such that for all x > c¢, 
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a >a. Putting together these two estimates shows lim,_,)- net =A. 


b) This is quite straightforward and left to the reader.+ 


Perhaps you were expecting the additional hypothesis lim,_,,- f(a) = +00 in con- 
dition (ii). As the proof shows, this is not necessary. But it seems to be very risky 
to present the result to freshman calculus students in this form! 


n 


EXAMPLE 7.1. We claim that for all n € Z*, limz_,., tz = 0. We show this 
by induction on n. First we don = 1: limgoo & = 0. Since limz.. g(x) = 0 


and limy-+00 fe) = limg-s00 s = 0, condition (ii) of L’Hépital’s Rule applies so 


limysoo a = 0. Induction Step: letn € Z* and suppose limg_+o. tz = 0. Then 


lim soo =e = SS img jog MAEVE = (+ 1) (limi 00S) = (n+ 1)-0=0. 


Why do calculus instructors not like L’H6pital’s Rule? Oh, let us count the ways! 


0 


1) Every derivative f’(x) = limp-+o fleth)-i@) is of the form 5. Thus many calcu- 


lus students switch to applying L’Hopital’s Rule instead of evaluating derivatives 
from the definition. This can lead to painfully circular reasoning. For instance, 


sine? Well, both numerator and denominator approach 0 and 


lim,z_+0 (ene = limz,9 “f= = cos0 = 1. What’s wrong with this? Well, how 
do we know that (sinx)’ = cosx? Thinking back, we reduced this to computing 


the derivative of sinz at x = 0, i.e., to showing that lim, ohh = 1! 


what is lim,_,o9 


2) Many limits which can be evaluated using L’Hépital’s Rule can also be eval- 
uated in many other ways, and often just by thinking a bit about how functions 
actually behave. For intance, try to evaluate the limit of Example 7.1 above without 
using L’Hopital. There are any number of ways. For instance: 


LEMMA 7.2 (Racetrack Principle). Let f,g : [a,oo) > R be two differentiable 
functions such that f’(x) > g(x) for alla >a. Then: 


a) We have f(x) — f(a) > g(x) — g(a) for allz >a. 
b) If f’(x) > g'(x) for allx > a, then f(x)— f(a) > g(x)—g(a) for allx >a. 


Proor. Put h= f—g:[a,~) OR. 
a) Then h’(x) > 0 for all x > a, so h is weakly increasing on (a, 00), and thus being 
continuous, weakly increasing on [a, co): for all x >a, f(x)—g(ax) = h(x) > h(a) = 
f(a) — g(a), and thus f(a) — f(a) > 9(2) — g(a). 
b) This is the same as part a) with all instances of > replaced by >: details may 
be safely left to the reader. 


PROPOSITION 7.3. Let f : [a,co) > R be twice differentiable such that: 
(i) f’(a) > 0 and 
(ii) f(x) > 0 for allx >a. 
Then limz-00 f(x) = co. 


PROoF. Let g be the tangent line to f at x = a, viewed as a function from 
[a,co) to R. Because f” = (f’)’ is non-negative on [a,oo), f’ is weakly increasing 


Nn fact, [R] states and proves the result with lim, _,,+ instead of lim,_,,—. I recast it this 
way since a natural class of examples concerns limz-+o0. 
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on [a, co), and thus for all x > a, 
f'(@) 2 f'(a) = g(a). 
Applying the Racetrack Principle, we get that for all x > a, 
f(x) — f(a) 2 g(x) - g(a), 
or 


f(@) = g(x) + F(a) — g(a) = g(a). 


Since g is a linear function with positive slope, 


ee a oa 


(i) Let fala )= x. One can therefore establish limz_,.. f= = co by showing that 
fi (x), f(x) are both positive for sufficiently large x. It is easy to see that f(x) > 0 
for all 2 > n. The analysis for f/’ is a bit messier; we leave it to the reader and try 
something slightly different instead. 
(ii) Since f/ (x) > 0 for all « > n, f is eventually increasing and thus tends either 
to a positive limit A or to too. But as x > co, r+ 1 > c&, so 
a+1 © 
A= lim — =e lim =eA. 
LOO (a + 1)” LOO (x + 1)” 

The only A € (0, 00] which satisfies A = eA is A = 00. 
(iii) Take logarithms: if A = limz-.c. a then 


log A = lim log = = lim x—nlogz. 
ee Xero) XL xw~—00 
Now if I(x) = x —nloga, then l/(z) = 1—4, 1" (a) = 3; both are positive for large 
enough x, so by Proposition 7.3, log A = oo and thus A = oo. 
(iv) When we learn eon Taylor series we will have access to a superior expression 
for e”, namely 7° 9 ;1: From this the desired limit follows almost immediately! 


3) The statement of L’H6pital’s Rule is complicated and easy for even relatively 

proficient practitioners of the mathematical craft to misremember or misapply. A 

classic rookie mistake is to forget to verify condition (i) or (ii): of course in general 

x "(ax 

lim f(z) ) # lim FX ), 
ein g(a) 7 234 g(a) 

try a random example. But there are subtler pitfalls as well. For instance, on 

F(z) _ fia) 


aay A need not imply that lim, 4, 4 Fay 


exists, so you cannot use L’Hopital’s Rule to show that a limit does not exist. 
EXAMPLE 7.2. Let f,g: RR by f(x) = 2? sin(+ ) and f(0) =0 and g(x) =a 

Then f and g are both differentiable (for f this involves going back to the limit 

definition of the derivative at x = 0 — we have seen this example before), and 


(a) _ 


lim,z-s9 rey aS lim,-+0 @ sin( t)= = 0. However, 


: 1 1 1 
lian ii) = lim 2z sin ( ) cos ( ) = — lim cos (=) . 
«30 g'/(%) = «30 x x «0 x 


and this limit does not exist. 


under conditions (i) and (ii), lima 
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Nevertheless, every once in a while one really does need L’H6pital’s Rule! We will 
encounter such a situation in our study of Taylor polynomials. And, you know, 
everyone else is doing it, so you should at least know how to do it... 


EXERCISE 7.1. Use L’Hopital’s Rule to give a short proof of Theorem 5.32. 


2. Newton’s Method 
2.1. Introducing Newton’s Method. 


Newton’s Method is an important procedure for approximating roots of differen- 
tiable functions. Namely, suppose that y = f(x) is a differentiable function and 
that we know — perhaps by the methods of calculus! — that there is a real number 
c such that f(c) = 0. Well, in general there are many, but suppose we restrict our- 
selves to some small interval in which we believe there is a unique root c. Say we 
do not need to know c exactly, but that we wish to approximate to any prescribed 
degree of accuracy — e.g. we may need to know its value to 100 decimal places. 

The first key idea is that Newton’s method is one of sucessive approxima- 
tions. That is, we start with a number x; which is an “approximate root” of f, 
ie., f(x1) is rather close to 0 (this is not a precise mathematical statement, but 
the closer x; is to the true root c, the better the method will work. If it’s too far 
away, then it won’t work at all.) Then we perform some ameloriation procedure 
resulting in a second approximate root x2, which is (in general) closer to the true 
root c than 2; is. And then we continue: performing our amelioration procedure 
again we get x3, performing it a third time we get x4, and so forth, resulting in an 
infinite sequence of approximate roots {x,,}°21. 


This amelioration procedure is very geometric: let n € Zt, and start with the 
approximate root x,. What we do is consider the tangent line |,,(a) to y = f(a) at 
X= 2X,. The equation of this line is 
y— f (an) = f eae -- Ln); 
so 
y=l(2)= fa) +f Calle - 2a): 
Now we take x,,41 to be 2z-intercept of the line l,,(x), i-e., the unique number such 


that l,(@n+41) = 0. So let’s do it: 
0= In(@n41) i f(%n) Bi f'(tn)(@n41 _ Ln); 


7 Se —f(&n) 
vik . f'(tn) 
a lon) 


En41 = In — filx i 
n 


Note that our expression for #41 is undefined if f’(2,) = 0, as well it should be: 
if the tangent line at x, is horizontal, then either it coincides with the x-axis (in 
which case x, is already a root of f and no amelioration is needed) or it is parallel 
to the x-axis, in which case the method breaks down: in a sense we will soon make 
precise, this means that x, is “too far away” from the true root c of f. 
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2.2. A Babylonian Algorithm. 


We can use Newton’s method to approximate 2. Consider f(x) = x? —2; straight- 
forward calculus tells us that there is a unique positive number c such that f(c) = 2 
and that for instance c € [1,2]. We compute the amelioration formula in this case: 


g2—-2 2? —(22-2) 2242 > ( >) 
= = = Int ; 


(27) Chery SEG, 


Wn Wn Wn 2 


In 


In other words, to get from x, to %,41 we take the average of x, and 2. 


In this case we are allowed to take any x, # 0; if we want to approximate the 
positive root of 2? — 2 = 0, it is more than plausible that we should start with a 
positive number. So let’s try 


t= 1 
Then 
ee Cd eae 
saa i a 
1/3. 4\ 17 
= +-)=— = 1.41666... 
va Fy € +) 12 oe 
1/17  24\ 577 
= = —" = 1.414215686.... 
ae (3s 7) 408 ote 
1 (577 816 
==(24=°)-= 470832 = 1.41421356237468991 . ... 
ts = 5 ‘ TES = 665857 /47083 35623746899 
1 (470832 1331714 
= = 1 27013566048 = 1.414213562 4880... 
ts = 5 (= —) 886731088897 /627013566048 356237309504880 


If I now ask my laptop computer to directly compute V2, then it tells me? that 
V2 = 1.414213562373095048801688724.... 


Thus x5 is accurate to 11 decimal places and 2¢ is accurate to 23 decimal places. 
Looking more carefully, it seems that each iteration of the “amelioration process” 
Ln > Ln41 roughly doubles the number of decimal places of accuracy. If this holds 
true, it means that the approximations get close to the true root very fast — it we 
wanted /2 to 100 decimal places we would only need to compute 29. 

The formula (27) for successive approximations to V2 was known to the an- 
cient Babylonians, thousands of years before tangent lines, calculus of any kind, 
and Isaac Newton. So in practice Newton’s method seems to work very well indeed. 


Similarly, for any a > 0, using Newton’s method as above with f(r) = x? — a 


leads to the recursion 
1 a 
Intl =F (2+ =) ry 
2 Ley 


application of which with x, = 1 leads to fantastically good numerical approxima- 
tions to /a. If you ever find yourself on a desert island and needing to compute \/a 
to many decimal placees as part of your engineering research to build a raft that will 


2Of course one should not neglect to wonder how my computer is doing this computation. I 
don’t have access to the source code the software I used, so I don’t really know, but it is plausible 
that it is in fact using some form Newton’s method. 
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carry you back to civilization, then this is probably the method you should use. 
And now if anyone asks you whether “honors calculus” contains any practically 
useful information, you must tell them that the answer is yes! 


2.3. Questioning Newton’s Method. 


Of course we haven’t proven anything yet. Here are two natural questions: 


QUESTION 7.4. Let f : I > R be differentiable, and let c € I be such that 
f(c) =0. 

a) Is there some subinterval (c — 6,c + 6) about the true root c such that 
starting Newton’s method with any x1 € (c— 6,c+6) guarantees that the 
sequence of approximations {x,,} gets arbitrarily close to c? 

b) If the answer to part a) is yes, given some x, € (c—6,c +0) can we give 
a quantitative estimate on how close Xp, is to c as a function of n? 


Questions like these are explored in a branch of mathematics called numerical 
analysis. 


2.4. Introducing Infinite Sequences. We will give some answers to these 
questions. First, the business of the x,,’s getting arbitrarily close to c should be con- 
strued in terms of a limiting process, but one of a kind which is slightly different and 
in fact simpler than the limit of a real-valued function at a point. Namely, a real 
infinite sequence z,, is simply an ordered list of real numbers 21, 2%2,...,2n,---; 
or — slightly more formally, is given by a function from the positive integers Z* to 
R, say f(n) = ay. If L € R, we say the infinite sequence {z,,} converges to L — 
and write z, — L — if for all « > 0 there exists N € Z*+ such that for alln > N, 
|x, — L| < «. This is precisely the definition of lim, f(x) = L except that our 
function f is no longer defined for all (or all sufficiently large) real numbers but 
only at positive integers. So it is a very close cousin of the types of limit operations 
we have already studied. 


Here is one very convenient property of limits of sequences. 


PROPOSITION 7.5. Let {%}°2, be a sequence of real numbers, and let f : R 
R be a continuous function. Suppose that x, > L. Then f(a,) > f(L). 


PROOF. Fix « > 0. Since f is continuous at L, there exists 6 > 0 such that 
jc—L| <6 => |f(x)—f(L)| < €. Moreover, since x, + L, there exists a positive 
integer N such that for all n > N, |x, — L| < 6. Putting these together: ifn > N 
then |x, — L| < 4, so |f(a,) — f(L)| < e. This shows that f(z,) > L. 


Remark: a) Proposition 7.5 is a close cousin of the fact that compositions are con- 
tinuous functions are continuous, and in particular the proof is almost the same. 
b) At the moment we are just getting a taste of infinite sequences. Later in the 
course we will study them more seriously and show that Proposition 7.5 has a very 
important converse: if f : R — R is a function such that whenever x, + DL, we have 
also f(@,) > f(L), then f is continuous. Thus preservation of limits of sequences 
is actually a characteristic property of continuous functions, and this suggests (cor- 
rectly!) that sequences can be a powerful tool in studying functions f : R > R. 


Thus Newton’s method starts with «, € R and produces an infinite sequence {z,, } 
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of “successive aproximations” , and our first question is whether — or more precisely, 
when, i.e., for which choices of x, — this sequence converges to the true root c. 


2.5. Contractions and Fixed Points. 
Recall that a fixed point of f : J > Ris a point c € I with f(c) =c. 


A function f : I — R is contractive (or is a contraction) if there is a < 1 
such that for all x,y € I, |f(x) — f(y)| < alz — y|; a real number a for which such 
an inequality holds will be called a contraction constant for f. 


EXERCISE 7.2. a) Which functions f : I — R have contraction constant 
0? 
b) Show that every contraction f : I > R is continuous. 


PROPOSITION 7.6. Let f : I > R be differentiable, and suppose there is a < 1 
such that for all x € I, |f’(a)| <a. Then a is a contraction constant for f. 


ProoF. Let « < y € I. By the Mean Value Theorem, there is c € (x,y) such 


that 
f(z) = fy) 
et 


f(x) — FMI = If" Olle — yl < ala — yl. 


LEMMA 7.7. A contractive function f : I > R has at most one fixed point. 


= f'(c), 


sO 


PRooF. Suppose there are 71 # 2 in I with f(x1) = 2 and f(r2) = x2. Let 
a < 1 be acontraction constant for f. Then |x| —22| = |f(x1)—f(x2)| < alv1—29; 
dividing through by |#1 — x2| gives 1 < a: contradiction. 


Suppose f : J > R is a contraction with constant a, and let c € I be a fixed point 
of f. Then for any 6 > 0, if « € [c—6,c + 4], then 


If(z) —e = |fe) — f()| S ale — ee < |Ja—el <4, 


so f :[c—6,c+6] > [c—46,c+]. This is a key point, because given any f : I > I 
and any x; € [c— 6,c + 6] C I, we may define a sequence 


1,02 = f(x1),€3 = f(t2) = f(F(@1)), +++. tn41 = f@n) = (fo---0 f)(@1),--- 


We call this the sequence of iterates of x; under f. 


EXAMPLE 7.3. Let a € R, and consider the function 
f: ROR, f(x) =az. 


Let us study the dynamics of iteration of this very simple function. 

Step 1: What are the fixed points? We set c = f(c) =ac. Thenc =0 is a fixed 
point no matter what a is. Conversely, if c #4 0 then we divide through to geta = 1, 
50 if a #1 then 0 is the only fixed point. Finally, if a = 1 then every c € R is a 
fixed point, i.e., f is the identity function. 

Step 2: Let us try to figure out the limiting behavior of the sequences of iterates. 
First observe that for any fixed point c, the sequence of iterates will be constant: 


OCC CGaes 


132 7. DIFFERENTIAL MISCELLANY 


80 of course it will converge to c. So really we are interested in the case when 
x, is not a fixed point of f. In fact we can - and this is an unusual feature 
arising because of the very simple f we choose — give an explicit formula for the 
general term xy, in the sequence of iterates. Starting at x, we have x2 = aa, 
3 = AL = a(azr,) = a?21, and so on: in general we have tp = a”—1x4. 

Case 1: If a = 0, then no matter what x, is, for alln > 2, x, = 0. Thus the 
sequence is eventually constant, with eventual value the unique fixed point: 0. 
Case 2: If a = 1, then for all n, t, = x1, 80 the sequence is constant, and this 
constant value is a fixed point of f. 

Case 3: If 0 < |a| <1, then a = a"~!ax, > 0. Here, no matter what the initial 
point x1 1s, the sequence of iterates converges to the unique fired point c = 0. 

Case 4: If 1 < lal, then for all 2, # 0, |xn| = |a®~!21| — oo: the sequence of 
iterates grows without bound, in fact with exponential speed. In particular it does 
not converge. One can say more about the signs if desired: e.g. when x, > 0 and 
a> 1, then every xp, is positive and the sequence of iterates approaches oo, whereas 
if x, > 0 anda < —1, then the terms x, alternate in sign while increasing in 
absolute value, so they do not approach either co or —oo. 


Here are three observations about the above example: 


1) If 21 is a fixed point of f, then the equence of iterates is constant. 

2) Whenever the sequence of iterates converges, it converges to a fixed point of f. 
3) The sequence of iterates converges for all initial points 7; € R iff Ja] < 1. But 
note that | f(x) — f(y)| = lal|z — y|, so f is contractive iff Ja] < 1. 


The first observation clearly holds for the iterates of any function f : J > I. 
What about the last two observations? In fact they also hold quite generally. 


LEMMA 7.8. Suppose f : I > I is continuous, 11 € I, and the sequence of 
iterates {x,} of x1 under f converges to L € I. Then L is a fixed point of f. 


PRooF. Since f is continuous and x, — L, by Proposition 7.5 we have f(t) > 
f(L). But f(an) = tn4i1, and if z, — LD then certainly t,41 > L as well. A 
sequence can have at most one limit, so f(Z) = L. 


LEMMA 7.9 (Contraction Lemma). For c € R and 6 > 0, put I = [e—6,c+ 9]. 
Suppose that f : I > R is a contraction with constant a and that f(c) =. 
a) For alla EI, f(x) él, ie, f: Tol. 
b) For any x, € I, define a sequence {@,}°21 by Un41 = f(Xn) for alln > 1. Then 
for alln € Zt, we have |tn41 — cl < a”™|a1 —c|. In particular tp, > c. 


PROOF. a) This was established above. We repeat the statement here because 
it is a key point: in order to be able to iterate x; under f we need f : J —> I. 


b) We compute |an41 — ¢] = |f(@n) — FCO) S alan — e| = al f@n-1) — FO) 


< a? |an—1 —cl= a? |an—2 —c 


=...=a"|r1 — cl. 
2.6. Convergence of Newton’s Method. 
Believe it or not, we are now very close to a convergence theorem for the sequence 


of iterates generated by Newton’s method. Let f : I > R be a C? function — ice., 
the second derivative exists and is continuous — and let c be a point of the interior 
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of I such that f(c) = 0 and f’(c) #0—asimple root. Note that since f’(c) > 0, f 
is increasing through c, so there exists 6 > 0 such that f is negative on [c—06,c) and 
positive on (c,c+ 46]. In particular c is the unique root of f on [ce — 6,c+ 6]. What 
we want to show is that — possibly after shrinking 6 — for any choice of initial ap- 
proximation x; € [c—6,c+6], the Newton’s method sequence converges rapidly to c. 


So...what does the material we have just developed have to do with this prob- 
lem? At first sight it does not seem relevant, because we have been talking about 
fixed points and are now interested in roots, we have been talking about iterating 
functions on [c—6,c+ 6] and we probably cannot iterate f on this interval: f(c) = 0 
and 0 need not lie in [c — 4, 6], and we have been talking about contractions and f 
need not be a contraction. What gives? 


The answer is that all of our preparations apply not to f but to some auxiliary 
function defined in terms of f. To figure out what this function is, consider the 
recursive definition of the Newton’s method sequence: 


En41 = In — filx y 
n 


Thus the sequence {2,}?2, is generated by repeatedly applying a certain func- 
tion...it’s just not the function f. Rather it is the amelioration function 
_ f(x) 

f(x) 
Now we have to check that our setup does apply to T, quite nicely. First, observe 
that a point x is a root of f if and only if it is a fixed point of T. Since by our 
assumption c is the unique root of f in [c — 6,c+ 6], c is the unique fixed point of 
T on this interval. 

The next order of business is to show that T is contractive, at least in some 

smaller interval around c. For this we look at the derivative: 


(a) 1 LOL) = Fa) F"(@) _ FE), 
(f'(a))? (f"(a))? 
This is all kosher, since we have assumed f’ is nonzero on [c — 6,c + 6] and also 
that f is C?. In fact, since f is C?, T’ is continuous. But now a miracle occurs: 
T'(c) = 0. Since T” is continuous at c, this means that by making 6 smaller we 
may assume that |T’(x)| < a for all x € [c— 6,c+ 6], for any positive a we want! 
Thus not only can we make T contractive, we can make it contractive with any 
contractive constant a € (0,1) we want! Thus we get the following result. 


THEOREM 7.10. (Convergence of Newton’s Method) Let f : I > R be a C? 
function, i.e., f” exists and is continuous. Let c € R be such that f(c) = 0 and 


f'(c) £0. Fix a € (0,1). Then there exists 5 > 0 such that for alla, € [c—6,c+5], 


the Newton’s method sequence In+1 = Ln — Fiend is well-defined and converges 


T(x) =a 


rapdily to c in the following sense: for alln € Z*, |an41 —c¢| < a"|x1 — cl. 


PRoor. Fix a € (0,1). As above, since T’(c) = 0, there exists 6 > 0 such that 
a is a contraction constant for T’ on [c — 6,c + 6], and therefore, by Lemma 7.9a), 
T maps [c—6,c+ 6] back into [c—6,c+ 6], so for any x € [c—6,c+0] the sequence 
of iterates tn41 = T (xp) is well-defined, and note that it is precisely the Newton’s 
method sequence with initial approximation x,. By Lemma 7.9b), for all n € Z*, 
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|Onz1 — cl] < a"\a, — cl. So x, — c, and indeed it does so exponentially fast. In 
fact, if we take a = a and 6 < 1, the above estimate ensures that x, approximates 


c to within n decimal places of accuracy. 


2.7. Quadratic Convergence of Newton’s Method. 


Although Theorem 7.10 is a very satisfying result, it is far from the last word 
on Newton’s method. In fact, if we compare the proven convergence rate of Theo- 
rem 7.10 with the empirically observed convergence rate of f(x) = x? — 2, 7, = 1, 
we see that what we have proved is not as good as what we observed: in our exam- 
ple the number of decimal places of accuracy doubled with each iteration, i.e., is an 
exponential function of n, whereas we proved that the number of decimal places of 
accuracy must grow at least linearly with n. That’s a big difference! And in fact, 
if you look back at exactly what we proved, it seems to itself suggest that more is 
true: namely, we get exponential convergence, but the base of the exponential gets 
better and better as we get closer to the point, i.e., as we get farther out into the 
sequence of iterates. This is faster than exponential decay with any fixed base. 

I believe it should be possible to use the idea expressed at the end of the last 
paragraph to give a proof of the quadratic convergence of Newton’s Method. For 
this, an essential tool is Taylor’s Theorem With Remainder, an important 
result which we will unfortunately not treat until much later on. In fact it is possi- 
ble to give a short, elementary proof of the quadratic convergence which not only 
avoids Taylor’s theorem but also avoids our analysis of the contractive properties 
of the amelioration map T. Here it is. 


THEOREM 7.11 (Quadratic Convergence of Newton’s Method). Let f: [> R 
be twice differentiable, and let c € I° be a root of f. 
a) Suppose there are real numbers 6, A,B > 0 such that for all x € [c—6,c+ 6], 
|f’(x)| > A and |f"(x)| < B. For any xo € [c— 6,c+ 4], let {xy} be the Newton’s 
Method sequence with initial value xo. Then for alln € N, 


B 
(28) lrmgt el S Fltn — ef, 


b) If f” is continuous on I and f'(c) #0, then there are indeed 6, A,B > 0 as in 
the statement of part a), so that (28) holds for all xo € [c— 6,c+ 4]. 


Proor. a) (W. Miller) First, for a,b € R, we denote by |{a, }]| the set of all 
real numbers on the closed line segment from a to 6; precisely |[a, b]| = [a,b] ifa <b 
and |[a,b] = [b, a] if a > b. Similarly, let |(a,b)| = |[a, bl] \ {a,b} be the open line 
segment from a to b. 

Let xp € [c—d,c+4], and let {x,,} be the Newton’s Method sequence of iterates. 
It will be useful to rewrite the defining recursion as 

~f(@n) 


Wn EN, @n41 -— fn = —. 


ad eee) 
Apply the Mean Value Theorem to f on |[z,,,c]|: there is yn € |(@n,c)| such that 


f (an) i f (tn) a fe) 


In —C In —C 


= f' (Yn). 
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Apply the Mean Value Theorem to f’ on |[z,Yn]|: there is zn € |(@n,Yn))| such 
that 

Tr — Yn 
The rest of the proof is a clever calculation: for n € N, we have 
—flen) | Fn) 
fi(tn) ~ f"(Yn) 
as hey = ao bs ae — NF" (@n) = a 


lant — ¢| = |(@n4t1 — tn) + (@n — ©)| = 


f!(an)f"(Yn) f' (an) f'(Yn) 
= f'(an) — f'n) |x | = ne |x y I|ar c| 
£'(Yn) ‘ EG 
f" (En) 2 oP 2 
— | F'n) [Zn — el” < qltn el". 


In the second to the last inequality above, we used the fact that since yp, lies between 
Ly and c, |%n — Yn| < [an — Cc]. 


b) This is left as an exercise for the reader. 


EXERCISE 7.3. Prove Theorem 7.11b). 


Let c be a real number, and let {x,,}92.9 be a sequence of real numbers. We say 
that the sequence {z,,} quadratically converges to c if 

(QC1) z, > c, and 

(QC2) There is C > 0 such that for all n € Z*, we have |an41 —c| < Clan — c|?. 


EXERCISE 7.4. a) Show that a sequence may satisfy (QC2) but not (QC1). 
b) Suppose that a sequence {xn} satisfies |ap—c| < min(1, 4). Show that 
{tn} quadratically converges to c. 
c) Deduce that under the hypotheses of Theorem 7.11, there is 6 > 0 such 
that for all xp € [c—6,c+ 6], the Newton’s Method sequence quadratically 
converges to c. 


EXERCISE 7.5. Let {x,} be a sequence which is quadratically convergent to 
cER. Viewing x, as an approximation to c, one often says that “the number of 
decimal places of accuracy roughly doubles with each iteration”. 


a) Formulate this as a statement about the sequence dy, = log) 9(|t%n — cl). 
b) Prove the precise statement you formulated in part a). 


2.8. An example of nonconvergence of Newton’s Method. 


Consider the cubic function f : RR R by f(x) = 2° — 2x +2, so 
xe —2t,+2 
3x2 — 2 


Take x, = 0. Then zg = 1 and x3 = 0, so the sequence of iterates will then 


In+1 = In — 


alternate between 0 and 1: one calls this type of dynamical behavior a 2-cycle. 
(The unique real root is at approximately —1.769, so we are plenty far away from it.) 


This example is the tip of a substantial iceberg: complex dynamics. If you 
consider cubic polynomials as functions from the complex numbers C to C, then 
you are well on your way to generating those striking, trippy pictures of fractal 
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sets that have been appearing on tee-shirts for the last twenty years. I recommend 
[Wa95] as an especially gentle, but insightful, introduction. 


3. Convex Functions 

3.1. Convex subsets of Euclidean n-space. 
Let n be a positive integer and let R” be Euclidean n-space, i.e., the set of 
all ordered n-tuples (41,...,%n) of real numbers. E.g. for n = 2 we get the plane 
R?, for n = 3 one gets “three space” R°, and for higher n the visual/spatial in- 
tuition is less immediate but there is no mathematical obstruction. For the most 
part R” is the subject of the sequel to this course — multivariable mathematics 


— but let’s digress a little bit to talk about certain subsets of R”. In fact for our 
applications we need only n = 1 and n = 2. 


Given two points P = (%71,...,%n), Q = (y1,---; Yn € R” we can add them: 
P+Q= (a1 +91,---,%n + Yn); 


a generalization of “vector addition” that you may be familiar with from physics. 
Also given any real number 4, we can scale any point P by \, namely 


AP = A(G1y ss Ln) =H Aw se Aaa): 
In other words, we simply multiply every coordinate of P by X. 


Let P # Q be points in R”. There is a unique line passing through P and Q. 
We can express this line parametrically as follows: for every \ € R, let 


Ry =(1-A)P+AQ = ((1— A)ar + Ay,.--, (1 — A)En + AYn)- 
In particular Ro = P and R; = Q. Thus the line segment PQ is given as the set 
{Ry |O<A< I}. 


Now here is the basic definition: a subset 2 C R” is convex if for all P,Q € O, the 
line segment PQ is contained in 2. In other words, if our physical universe is the 
subset 2 and we stand at any two points of 2, then we can see each other: we have 
an unobstructed line of view that does not at any point leave 2. There are many 
convex subsets in the plane: e.g. interiors of disks, ellipses, regular polygons, the 
portion of the plane lying on one side of any straight line, and so forth. 


EXERCISE 7.6. Let Q1,...,Qn be conver subsets of R”. 


a) Show that (\_, 0; — the set of all points lying in every Q; — is convex. 
b) Show by example that U;_, Q: need not be convex. 


When n = 1, convex subsets are quite constrained. Recall we have proven: 


THEOREM 7.12. For a subset Q CR, the following are equivalent: 


(i) Q is an interval. 
(ii) Q is convex. 
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3.2. Goals. 


In freshman calculus one learns, when graphing a function f, to identify subinter- 
vals on which the graph of f is “concave up” and intervals on which it is “concave 
down”. Indeed one learns that the former occurs when f”(x) > 0 and the latter 
occurs when f(a) < 0. But, really, what does this mean? 

First, where freshman calculus textbooks say concave up the rest of the math- 
ematical world says convex; and where freshman calculus textbooks say concave 
down the rest of the mathematical world says concave. Moreover, the rest of the 
mathematical world doesn’t speak explicitly of concave functions very much be- 
cause it knows that f is concave exactly when —/f is convex. 

Second of all, really, what’s going on here? Are we saying that our definition 
of convexity is that f” > 0? If so, exactly why do we care when f” > 0 and when 
f” <0: why not look at the third, fourth or seventeenth derivatives? The answer is 
that we have not a formal definition but an intuitive conception of convexity, which 
a good calculus text will at least try to nurture: for instance I was taught that a 
function is convex (or rather “concave up”) when its graph holds water and that 
it is concave (“concave down”) when its graph spills water. This is obviously not 
a mathematical definition, but it may succeed in conveying some intuition. In less 
poetic terms, the graph of a convex function has a certain characteristic shape that 
the eye can see: it looks, in fact, qualitatively like an upward opening parabola or 
some portion thereof. Similarly, the eye can spot concavity as regions where the 
graph looks, qualitatively, like a piece of a downward opening parabola. And this 
explains why one talks about convexity in freshman calculus: it is a qualitative, 
visual feature of the graph of f that you want to take into account. If you are 
graphing f and you draw something concave when the graph is actually convex, 
the graph will “look wrong” and you are liable to draw false conclusions about the 
behavior of the function. 

So, at a minimum, our task at making good mathematical sense of this portion 
of freshman calculus, comes down to the following: 


Step 1: Give a precise definition of convexity: no pitchers of water allowed! 


Step 2: Use our definition to prove a theorem relating convexity of f to the second 
derivative f”, when f” exists. 


In fact this is an oversimplification of what we will actually do. When we try 
to nail down a mathematical definition of a convex function, we succeed all too 
well: there are five different definitions, each having some intuitive geometric ap- 
peal and each having its technical uses. But we want to be talking about one class 
of functions, not four different classes, so we will need to show that all five of our 
definitions are equivalent, i.e., that any function f : J — R which satisfies any one 
of these definitions in fact satisfies all four. This will take some time. 


3.3. Epigraphs. 


For a function f : I + R, we define its epigraph to be the set of points (x, y) € xR 
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which lie on or above the graph of the function. In fewer words, 


Epi(f) = {(@@,y) € 1 xR] y 2 f(a)}. 


A function f : I > R is convex if its epigraph Epi(f) is a convex subset of R?. 
Example: Any linear function f(z) = mz + b is convex. 
Example: The function f(a) = |a| is convex. 


Example: Suppose f(x) = az? + ba +c. Then Epi(f) is just the set of points 
of R? lying on or above a parabola. From this picture it is certainly intuitively 
clear that Epi(f) is convex iff a > 0, ie., iff the parabola is “opening upward”. But 
proving from scratch that Epi(f) is a convex subset is not so much fun. 


3.4. Secant-graph, three-secant and two-secant inequalities. 
A function f : I > R satisfies the secant-graph inequality if for alla <beETl 
and all A € [0, 1], we have 
(29) f((L=A)a + Ad) < (1 = A) f(a) + AF (0). 


As \ ranges from 0 to 1, the expression (1 — A)a + Xb is a parameterization of the 
line segment from a to b. Similarly, (1 — A) f(a) + Af(b) is a parameterization of 
the line segment from f(a) to f(b), and thus 


AH ((1—A)a + AB, (1 — A) f(a) + AF (B)) 


parameterizes the segment of the secant line on the graph of y = f(x) from 
(a, f(a)) to (b, f(b)). Thus the secant-graph inequality is asserting that the graph 
of the function lies on or below the graph of any of its secant line segments. 


A function f : I > R satisfies the three-secant inequality if for alla <a <b, 


f@) = $@) - fO=F@) - SO=-S@) 


ra a b-a = b-2 


(30) 


A function f : I > R satisfies the two-secant inequality if for alla < x < b, 


(31) f(z) = £@ © FO) -f@ 


r—a b-—a 
PROPOSITION 7.13. For a function f : I — R, the following are equivalent: 
(i) f satisfies the three-secant inequality. 
(ii) f satisfies the two-secant inequality. 
(itt) f satisfies the secant-graph inequality. 
(iv) f is conve, i.e., Epi(f) is a convex subset of R?. 


PROOF. We will show (i) (ii) (iii) (i) and (iii) <== (iv). 
(i) — > (ii): This is immediate. 
(ii) <> (iii): The two-secant inequality 


f@) = f@) - )=t@ 


w—-a —a 
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is equivalent to 


= La,p(2), 


say. Now L,,4() is a linear function with L,4(a) = f(a) and L,4,b() = f(b), hence 
it is the secant line between (a, f(a)) and (0, f(b)). Thus the two-secant inequality 
is equivalent to the secant-graph inequality. 

(iii) ==> (i): As above, snce the secant line L,,(x) from (a, f(a)) to (6, f(b)) has 


equation y = f(a) 4 (222) (x — a), the secant graph inequality implies 
fle) — fla) - FO) ~ Ha) 


ca b-a 


To get the other half of the three-secant inequality, note that we also have 


Las(e) = F0) + $9-FOG 29, 


and the inequality f(a) < f(b) + BORON — 2) is easily seen to be equivalent to 
f(b) — fla) — 0) — fle) 
b-—a 7 b-x — 
(iii) — > (iv): Let Py = (x1, y1), Po = (x2, y2) € Epi(f). We want to show Epi(f) 
contains the line segment joining P; and Pj. This is clear if 7] = x2 = x — in 
this case the line segment is vertical, since since y; and y2 are both greater than or 
equal to f(a), so is every point y in between y; and yo. So we may assume x 4 2 
and then that «1 < x2 (otherwise interchange P; and P2). Seeking a contradiction, 
we suppose there is \; € (0,1) such that (1 — 1) P; + Ai Po ¢ Epi(f): that is, 
(1—Ai)yi + Arye < f((1 — Ar)ar + Arx2). 
But since f(1) < y and f(x%2) < yo, we have 


(1 — Ai) f (a1) + Ar f(e2) < (1 — Ardy + Arye 


and thus 

(1 — Ar) f (a1) + Arf(e2) < f((1 — Ax)@1 + Arxa), 
violating the secant-graph inequality. 
(iv) = > (iii): Let « < y € I. Since (a, f(a)) and (y, f(y)) lie on the graph of f, 
they are elements of the epigraph Epi(f). Since Epi(f) is convex the line segment 
joining (a, f(x)) and (y, f(y)) lies inside Epi(f). But this line segment is nothing 
else than the secant line between the two points, and to say that it lies inside the 
epigraph is to say that the secant line always lies on or above the graph of f. 


COROLLARY 7.14. (Generalized Two Secant Inequality) Let f : I > R be a 
convex function, and let a,b,c,d€ I witha<b<c<d. Then 
f= f@) — f@=-fO 
b-a ~— d-c ~° 
ProoF. If 6 = c then (32) is the Two Secant Inequality, so we may suppose 
b<c. Applying the Two Secant Inequality to the points a, b,c gives 
f(b) - Fla) — fle) - F) 
b-—a = c—b 


(32) 
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and applying it to the points b, c,d gives 


fle) - #00). fl) - 0 
c—b a d-c — 


Combining these two inequalities gives the desired result. 


Exercise (Interlaced Secant Inequality): Let f : J + R be a convex function, and 
leta<b<c<deé€éT. Show that we have 


c-a mn d—b 


3.5. Continuity properties of convex functions. 


THEOREM 7.15. Suppose f : I > R is convex, and let [a,b] be any subinterval 
of I°. Then f is a Lipschitz function on [a,b]: there exists a constant C' such that 


for all z,y € [a,6], |f(x) — fly)| < Cla — yl- 


PrRooF. Choose u,v,w,z € I° with u<v<aandb<w < z. Applying the 
Generalized Two Secant Inequality twice we get 


fo) =f) - FW)-f@) © F=f) 


v-—U = y-ax % zZ—-Ww 
Thus 
IM@) = FOI Spe (FO = FH fe) 10) 
jz —y| v-U , zZ-W , 
(ee (O20) Homo) 


is a Lipschitz constant for f on [a, }]. 


COROLLARY 7.16. Let I be any open interval, and let f : I + R be a convex 
function. Then f is continuous. 


Exercise: Prove Corollary 7.16. 


It turns out that — at least according to our definition — a convex function need 
not be continuous on an endpoint of an interval on which it is defined. The follow- 
ing result analyzes endpoint behavior of convex functions. 


PROPOSITION 7.17. Leta € R, and let I be an interval of the form (a,b), (a, }] 
or (a,co). Let f : I + R be conver. 
a) lim, a+ f(x) € (—00, oo]. 
b) Suppose that L = lim,_,,+ f(x) < oo. For M € R, define a function f : {a}UI 3 
R by f(a) = M, f(x) = f(x) forx € I. Then f is convex iff M > L and continuous 
ata iff M=L. 


Exercise: Prove Proposition 7.17. 
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3.6. Differentiable convex functions. 


THEOREM 7.18. For a differentiable function f : I > R, TFAE: 
(i) f ts conver. 
(it) f’ is weakly increasing. 


PrROoF. For both directions of the proof it is convenient to consider “fixed” 
a<beéTJand “variable” a < x < b. 
(i) = > (ii): For x € (a,b], we define 


and for x € [a, b) we define 


FQ) = fe) 


b-2 


S(a) = 
Since f is convex, the three-secant inequality for a < x < b holds: 
s(x) < s(b) = S(a) < S(a). 
Taking limits, we get 


f’(a) = lim s(x) < s(b) = S(a) < lim S(x) = f'(b). 
zat t—b— 
(ii) = > (a): Let g(x) = fe) Fa) We will show that g is increasing on (a, 6]: this 
gibes the two-secant inequality and thus the convexity of f. Since f is differentiable, 
so is g and 


(a) = E=oFe)— le) - fle). 


By the Mean Value Theorem, f(a) Fa) = f'(y) for some a < y < «. Since f" is 
increasing, 


equivalently 
(x — a) f"(x) — (f(x) — f(a)) 2 0. 


Thus g/(x) > 0 for all x € (a, 6], so indeed g is increasing on (a, 6]. 


COROLLARY 7.19. A differentiable convex function is C?. 


ProoFr. By Theorem 7.18, f’ is weakly increasing so has only jump disconti- 
nuities. By Darboux’s Theorem, a derivative cannot have jump discontinuities. 


COROLLARY 7.20. (Kane? Criterion) For twice differentiable f : I > IR, TFAE: 
(i) f ts conver. 
(ii) f” > 0. 

Proor. By Theorem 7.17, f is convex iff f’ is weakly increasing. Moreover, 
f’ is weakly increasing iff f” = (f’)’ > 0. We’re done. 


3 Andrew Kane was a student in the 2011-2012 course who suggested this criterion upon being 
prompted in class. 


142 7. DIFFERENTIAL MISCELLANY 
3.7. An extremal property of convex functions. 


The following result secures the importance of convex functions in applied mathe- 
matics: for these functions, a critical point must be a global minimum. 


THEOREM 7.21. Let f : I > R be a differentiable convex function. Let c € I 
be a stationary point: f’(c) =0. Then: 
a) f attains its global minimum at c. 
b) Unless f is constant on some nontrivial subinterval of I, f attains a strict global 
minimum atc: for allx #c, f(x) > f(c). 


ProoF. By Theorem 7.18, f’ is weakly increasing. 

a) Suppose d > c is such that f(d) < f(c). Then the secant line between 
(c, f(c)) and (d, f(d)) has negative slope, so by the Mean Value Theorem there is 
z € (c,d) such that f’(z) <0. But then c < z and 0= f’(c) > f’(z), contradicting 
f’ being weakly increasing. So f(c) < f(x) for all x >. 

Similarly, suppose b < c is such that f(b) < f(c). Then the secant line between 
(b, f(b)) and (ce, f(c)) has positive slope, so there is w € (b,c) such that f’(w) > 0. 
But then 6 < c and f’(b) > 0 = f'(c), contradicting f’ being weakly incereasing. 
So f(x) > f(c) for alla <c. That is, f attains a global minimum at c. 

b) Ife 4 c is such that f’(e) = 0, then since f’ is weakly increasing, f’ = 0 on 
the subinterval |[c, e]| from c to e, and thus f is constant on |[c, e]|. 


Exercise: a) Let f : (a,b) — R be a convex function such that lim,_,,+ f(a) = 
lim,_,)- f(@) = oo. Show that f assumes a global minimum. 

b) State and prove an analogue of part a) with (a,b) replaced by R = (—o0, ov). 
c) Exhibit a convex function f : R > R which does not assume a global minimum. 


3.8. Supporting lines and differentiability. 


Let f : J — R be a function. A supporting line for f at c € I is a linear 
function ¢: IR > R such that f(c) = €(c) and f(x) > (x) for all x € I. 

Example: Consider the function f : R — R given by f(x) = x?. Observe that 
the horizontal line @ = 0 is a supporting line at c = 0: indeed f(0) = 0 and 
f(x) => 0 for all x. Notice that ¢ = 0 is the tangent line to y = f(x) at c=0. 


Example: More generally, let A,B,C € R with A #0. We claim that for allc eR 
the tangent line ¢,(2) to the parabola f(x) = Ax? + Bx+C is the unique line pass- 
ing through (c, f(c)) such that f(x) # (a) for all a #c. To see this, consider the 
function g(a) = f(x) —€.(a). Then g is a quadratic polynomial with leading coeffi- 
cient A and g(c) = g/(c) = 0, so g(c) = A(a—c)?, and thus f(x) — ¢.(x) = g(x) £0 
for « # c. On the other hand, let @ be any other line passing through (c, f(c)). 
Then h(x) = f(x) — &(a) is a degree two polynomial with h(c) = 0. Moreover, since 
é is not the tangent line at c, h’(c) € 0, and thus A has a simple root at c, i.e., 
h(x) = A(« — c)j(x) with j(x) a linear function with j(c) # 0. Therefore j has a 
root at some d # c and that point d, ¢(x) = f(x). 

We claim that the tangent line ¢,(x) to f(x) = Ax?+Bx+C is a supporting line 
iff A > 0. Indeed, on both the intervals (—oo, c) and (c, oo) the continuous function 
f(x)—£-(«) is nonzero, so must have constant sign. But limz-.+00 f()—c(x) = 00 
if A > 0 and —oo if A < 0. It follows that if A > 0, f(x) — &-(#) > 0 for alla Ac 
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— so £, is a supporting line — and also that if A < 0, f(a) — £.(x) < 0 for all x — so 
é. is not a supporting line. Note that since f’” (x) = 2A, f is convex iff A > 0. 


Example: Consider the function f : R > R given by f(x) = |a|. Since Epi(/) 
is a convex subset of R?, f is convex. For every c > 0, the line y = x is a sup- 
porting line, and for every c < 0, the line y = —~ is a supporting line, and in both 
cases these supporting lines are unique. For c = 0, y = 0 is a supporting line, but 
it is not the only one: indeed y = mz is a supporting line at c= 0 iff -l1<m<1. 
Note that the smallest slope of a supporting line is the left-hand derivative at zero: 
fO+h)- FO) _ ah 


! _ 4: : chan ee 
f-(0) = ne h 7 a h 1, 


and the largest slope of a supporting line is the right-hand derivative at zero: 


Pte tii OS a gia lh 


1. 
h>o+ h hot h 


LEMMA 7.22. Convex functions are closed under suprema. More precisely, if 
{fi: 1 Rhier is a family of convex functions, f : I > R is a function such that 
for allx EI, f(x) =supje, fi(x), then f is convex. 


Proor. Leta <be€J and \€ (0,1). Then 
f(A -—A)a+ Ab) = =D fi((1 — A)a + Ab) 
< (1—A)sup f(a) + Aes £0) = (1-A)f(a) + Af(). 


tet 


THEOREM 7.23. Let I be an open interval. For a function f : I 4 R, TFAE: 
(i) f ts conver. 
(it) f admits a supporting line at each c € I. 


PRooF. (i) = > (ii): Neither property (i) or (ii) is disturbed by translating 


the coordinate axes, so we may assume that c = 0 and f(0) = 0. Let a € J \ {0}. 
For all Ay, A2 > 0 such that A1a, —A2,a € I, by the secant-graph inequality we have 


a X 
O= (Ar + Aa) f (5 aoe A2@) 4 yi a (ua) S Ai f(—A2q) + A2f(A1@), 
or 

—fl(-r20) — fre) 
A2 — AL : 

It follows that sup), afk) < inf), Ame) so there is m € R with 

~fl-A2@) 2 Fe) 

A2 ee Ny 


Equivalently, f(ta) > mt for all ¢ € R such that ta € J. Thus ¢(7) = ma isa 
supporting line for f at c= 0. 

(ii) ==> (i): For each c € J, let €. : J > R be a supporting line for f at c. Since 
for all x € I, f(x) > €-(x) for all c and f(c) = é-(c), we have f(x) = sup,e; €-(2). 
Since the linear functions @, are certainly convex, f is the supremum of a family of 
convex functions, hence convex by Lemma 7.21 
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Before stating the next result, we recall the notion of one-sided differentiability: 
if f is a function defined (at least) on some interval [c — 6,c], we say f is left 
differentiable at c if lim,_,,- Ae) He) exists, and if so, we denote this limit by 
fi (c), the left derivative of f at c. Similarly, if f is defined (at least) on sme 
interval [c,c+06), we say f is right differentiable at c if lim,_,.+ f¢ fe) FC) Fle) 
and if so, we denote this limit by f/ (c), the right derivative of f at c. (As usual 
for limits, f is differentiable at c iff f’ (c) and f',(c) both exist and are equal.) 


exists, 


THEOREM 7.24. Let I be an interval, cE I°, and f : I +R be convex. 
a) f is both left-differentiable and right-differentiable atc: f'(c) and fi(c) exist. 


b) For allc € (a,b), fL() < fie). 
c) fl, f, : IR are both weakly increasing functions. 
d) A line € passing through (c, f(c)) is a supporting line for f iff its slope m satisfies 


fet = fe): 


PRroor. We follow [Go, p. 153] and [Gr, pp. 8-10]. 
a), b) Define y : (a,b) \ {c} > R by 


Further, put 

A= e((a, c)), B= ple, b). 
From the three-secant inequality we immediately deduce all of the following: ¢ is 
weakly increasing on (a,c), y is weakly increasing on (c,b), and A < B. Thus 


f_(c) = lim y(#) =sup A < inf B= lim y(z) = fi(c). 
@2—>cT a—et 
c) Let 21,72 € (a,b) with x1 < x2, and choose v with 71 < v < xg. Then by part 
b) and the three-secant inequality, 


V—2X4 = VU— x2 
d) The proof of parts a) and b) shows that for x € J, 
e)>fo+f_O@—o), ifa<e, 


2) =f) + f(la-0), fa >e 

Thus if f! (ce) <m < fi(c), we have 

+ fi (\(a—c) > fl) +m(a—o), ifu<c, 
> $0) + FL) — 0) = mae) if x > ¢, 


so (a) = f(c) + m(ax — c) is a supporting line for f at c. That these are the only 
possible slopes of supporting lines for f at c is left as an exercise for the reader. 


< fl (x2) < fi. (a2). 


Exercise: We supppose the hypotheses of Theorem 7.24, and for m € R, put 
Ue) = f(e) + (xo). 
a) Suppose that m < f! (c). Show that there is 6 > 0 such that for all x € (c—4,c), 
e(x) > f(x) and thus @ is not a supporting line for f at c. 
b) Suppose that m > fi! (c). Show that there is 6 > 0 such that for all  € (c,c+4), 
€(x) > f(x) and thus @ is not a supporting line for f at c. 


Exercise: Assume the hypotheses of Theorem 7.24. Show TFAE: 
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(i) f is continuous at c. 

(ii) f, is continuous at c. 

(iii) f is differentiable at c, and the tangent line is a supporting line at c. 

(iv) f has a unique supporting line at c. 

Remark: Because a weakly increasing function can only have jump discontinu- 
ities, it can be shown that such functions are continuous at “most” points of their 
domain. For those who are familiar with the notions of countable and uncountable 
sets, we may be more precise: the set of discontinuities of a monotone function 
must be countable. Since the union of two countable sets is countable, it follows 
that a convex function is differentiable except possibly at a countable set of points. 


Remark: One can go further: a convex function is twice differentiable at “most” 
points of its domain. The sense of “most” here is different (and weaker): the set 
of points at which f fails to be twice differentiable has measure zero in the sense 
of Lebesgue. This result, which is itself due to Lebesgue, lies considerably deeper 
than the one of the previous remark. 


3.9. Jensen’s Inequality. 


THEOREM 7.25. (Jensen’s Inequality) Let f : I + R be continuous and convex. 
For any %1,...,%n € I and any \1,...,An € [0,1] with Ay +... + An =1, we have 


PRooF. We go by induction on n, the base case n = 1 being trivial. So 
suppose Jensen’s Inequality holds for some n € Z*, and consider 21,...,%n41 € I 
and Ay,.--,;Anqi € [0,1] with Ay +...+ Angi = 1. If Anyi = 0 we are reduced 
to the case of n variables which holds by induction. Similarly if A,41; = 1 then 
Ay =... = An = 0 and we have, trivially, equality. So we may assume An,41 € (0,1) 
and thus also that 1 — A,41 € (0,1). Now for the big trick: we write 


v1 Xn 
Lo tAn nm+1 = (1-An t...4 Zn })+An meeT; 
MS eat y) (a ie ) Ser 
so that 
M1 An 
f Ae pastas An&n) = f(a Anti) Ly fete La) t Antitn41) 
1 = An4q1 1 = An+1 
M Xn : 
<(1—An botscoe| n} +An n+1): 
= ( +i)f (; ae fe ) +1f (@n41) 


=? aan a are non-negative numbers that sum to 1, by induction 
n+1 n+1 
the n variable case of Jensen’s Inequality can be applied to give that the above 


expression is less than or equal to 


Since 


An 
1— An+1 


(Anan) (—* Ce 


Xn+1 


F(en)) oh Anti (@n41) 


= Arf (#1) Peet Anf (%n) Te An+il (@n+41): 
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3.10. Some applications of Jensen’s Inequality. 


Example: For any p > 1, the function f : [0,co) — R by x + @? is twice dif 
ferentiable on (0,00) with f(x) > 0 there, so by the Kane Criterion f is convex 
on (0,00). Also f is continuous at 0, so f is convex on [0, 00). 


Example: The function f : R — R given by « + e* has f”(a) = e” > 0 for 
all x, so by the Kane Criterion f is convex on R. 


By plugging these convex functions into Jensen’s Inequality and massaging what 
we get a bit, we will quickly deduce some very important inequalities.* 


THEOREM 7.26. (Weighted Arithmetic Geometric Mean Inequality) Let 11,...,%n € 
[0, co) and 4,...,An € [0,1] be such that Ay +...+An =1. Then: 


(33) ce eegtn < yoy +... +AnZn- 
Taking 4, =...=An= 1, we get the arithmetic geometric mean inequality: 
Gee < Pree ae aa 
n 
PROOF. We may assume 21,...,%, > 0. For 1 <i<n, put y; = logaz;. Then 


AL oN 
ay see ann = closer tn") = eAryittAndn < \yeV 4. tne’ = Ayait...tAnTn- 


THEOREM 7.27. (Young’s Inequality) 
Let x,y € [0,c0) and let p,q € (1,00) satisfy : + ; =1. Then 
(34) ry < se + a 

Pp qd 

PrRooF. When either x = 0 or y = 0 the left hand side is zero and the right hand 
side is non-negative, so the inequality holds and we may thus assume x, y > 0. Now 
apply the Weighted Arithmetic-Geometric Mean Inequality with n = 2, x, = 2?, 
t2=y!, r»1 a dr2 = We get 


P q 
ry = (x?) > (y2)4 — apa}? < Ayr + Age = rt + a 


THEOREM 7.28. (Hélder’s Inequality) 
Let ©1,..-,2n,Y1,+--,Yn € R and let p,q € (1,00) satisfy 5 + ; =1. Then 


1 


1 
(35) Ierya| +... + |an¥n| S (lal? +--+. + len l?)? (lysl? +--+ [yn l?)¢ 
ProoF. As above, the result is clear if either 7] =...= 2%, =Oory, =... = 
Yn = 0, so we may assume that neither of these is the case. For 1 <i < n, apply 
Young’s Inequality with 
|x| _ yi 
ry T 
(vif? +... + |anl?)? (lysl? +--+ Iynl4) 4 


is 


’ 


4Unfortunately we will not see here why these inequalities are important: you’ll have to trust 
me on that for now. But just as soon as you take graduate level real analysis, you’ll see these 
inequalities again. 
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and sum the resulting inequalities from i = 1 to n, getting 
Dic lel og tel Sy. 
(Je1P +... + |anl?)? (yal? +--+ [wnlt)* P| 


THEOREM 7.29. (Minkowski’s Inequality) 
For £1,.--,;%n,Y1;---;Yn © R and p> 1: 


1 a 1 
(36) (Jer + ya[P +... + [2n + Ynl?)? S (la)? +... + [anlP)? + (ly? +--+ lyn?) ? 
PrRooFr. When p = 1, the inequality reads 


Izy + y| +... + [en + Yn] S leil + ll +... + [en] + [yn 
and this holds just by applying the triangle inequality: for all 1 <i <n, |ai+yi| < 
|x;| + |y:. So we may assume p > 1. Let q be such that 5 + ; = 1, and note that 
then (p — 1)q = p. We have 


x1 yi|? Feet |Z Yn|? 

N p-14 \ ! p-l4 p-14 N N p-1 y 

S |eaflar typ +... + [ellen + ynlP + [ya|fer y/o +--+ [Yn len +YmlP” S 
1 1 1 1 
(ler /P+..+lan|?)? (erty P+. Flan tynlP) +g P+. Lyn)? (lea tya P+. «Flan tYn|?) 4 

1 1 1 
= (lou? +--+ lend?) # + (lua? + fanlP)*) (laa tan? +--+ Jn yal) 
Dividing both sides by (|v1 + yi|? +... + |@n 4 YnlP)4 and using 1—¢ a 3, we get 


the desired result. 


COROLLARY 7.30. (Triangle Inequality in R") For alla1,...,0n,Y1,---,Yn ER 


Vai ty)? +--+ (en 4 Un? < y/23 +... +224 VR t... + y2. 


CHAPTER 8 


Integration 


1. The Fundamental Theorem of Calculus 


Having “finished” with continuity and differentiation, we turn to the third main 
theme of calculus: integration. The basic idea is this: for a function f : [a,b] > R, 


we wish to associate a number ie f, the definite integral. When f is non-negative, 


our intuition is that f f should represent the area under the curve y = f(x), or 
more precisely the area of the region bounded above by y = f(x), bounded below 
by y = 0, bounded on the left by « = a and bounded on the right by x = b. 

Unfortunately this is not yet a formal definition, because we do not have a 
formal definition of the area of a subset of the plane! In high school geometry 
one learns only about areas of very simple figures: polygons, circles and so forth. 
Dealing head-on with the task of assigning an area to every subset of R? is quite 
difficult: it is one of the important topics of graduate level real analysis: measure 
theory. 


So we need to back up a bit and give a definition of ee f. As you probably know, 


the general idea is to construe fe f as the result of some kind of limiting process, 
wherein we divide [a,b] into subintervals and take the sum of the areas of cer- 
tain rectangles which approximate the function f at various points of the interval 
(Riemann sums). As usual in freshman calculus, reasonably careful definitions 
appear in the textbook somewhere, but with so little context and development that 
(almost) no actual freshman calculus student can really appreciate them. 


But wait! Before plunging into the details of this limiting process, let’s take a 
more axiomatic approach: given that we want { f to represent the area under 
y = f(x), what properties should it satisfy? Here are some reasonable ones. 


(11) If f =C is a constant function, then Ee C=C(b—-a). 
(12) If fa(w) < fo(a) for all x € [a,b], then [? fir < J? fo. 
(13) fa<ec<b, then f’ f= fof + fff. 


Exercise 1.1: Show (I1) implies: for any f : [a,b] > R and any c € [a,b], ie f=0. 


It turns out that these three axioms already imply many of the other properties we 
want an integral to have. Even more, there is essentially only one way to define 


i f so as to satisfy (11) through (I3). 
Well, almost. One feature that we haven’t explicitly addressed yet is this: for 
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which functions f : [a,b] + R do we expect vi f to be defined? For all functions?? 
A little thought shows this not to be plausible: there are some functions so patho- 
logical that there is no reason to believe that “the area under the curve y = f(a)” 
has any meaning whatsoever, and there are some functions for which this area con- 
cept seems meaningful but for which the area is infinite. 


So it turns out to be useful to think of integration itself as a real-valued function, 
with domain some set of functions {f : [a,b] > R}. That is, for each a < b we 
wish to have a set, say Ra, b], of integrable functions f : [a,b] — R and for each 
f € Ria, b], we wish to associate a real number i f. As to exactly what this set 
Ra, b] of integrable functions should be, it turns out that we have some leeway, but 
to get a theory which is useful and not too complicated, let’s assume the following: 


(10) For all real numbers a < b: 
a) Every continuous f : [a,b] — R lies in Ra, 8). 
b) Every function f € R[a, b] is bounded. 


By the Extreme Value Theorem, every continuous function f : [a,b] > R is 
bounded. Thus the class C[a, 6] of all continuous functions f : [a,b] — R is con- 
tained in the class B[a, b] of all bounded functions f : [a, 6], and axiom (10) requires 
that the set of integrable functions lies somewhere in between: 


C[a, b] C Ra, b] C Bla, b]. 


Let’s recast the other three axioms in terms of our set R[a, b] of integrable functions: 


(11) If f = C is constant, then f € Ra, b] and r C=C(b-a). 

(12) If for fi, fo € R[a, b] we have fi(x) < fo(x) for all a € [a, b], then fe fi< ie fa. 
(13) Let f : [a,b] > R, and let c € (a,b). Then f € Ra, }] iff f € Rla,c] and 
f € Ric, b]. If these equivalent conditions hold, then iN f= se ft fe f. 


If this business of “integrable functions” seems abstruse, then on the first pass just 
imagine that R[a, b] is precisely the set of all continuous functions f : [a,b] > R. 


Now we have the following extremely important result. 


THEOREM 8.1. (Fundamental Theorem of Calculus) Let f € Rla,b| be any 
integrable function. For x € [a,b], define F(a) = J” f. Then: 
a) The function F : [a,b] > R is continuous at every c € [a, b. 
b) If f is continuous at c € [a,b], then F is differentiable at c, and F'(c) = f(c). 
c) If f is continuous and F is any antiderivative of f —i.e., a function F : [a,b] > R 
such that F’(x) = f(a) for all x € [a,b], then De = F(b) — F(a). 


Proor. By (10), there exists M € R such that |f(x)| < M for all x € [a, d]. If 
M =0 then f is the constant function 0, and then it follows from (I1) that F is 
also the constant function zero, and one sees easily that the theorem holds in this 
case. 

So we may assume M > 0. For all € > 0, we may take 6 = 77. Indeed, by (13) 


(37) Fay-Fe=f s-f r= fs. 


1. THE FUNDAMENTAL THEOREM OF CALCULUS 151 


Moreover, let a< A < B <b. Then still —M < f(x) < M for all x € [A, B], so by 
(12) and then (12) we have 


B B 
-M(B-A)= (M)< [ f < M(B- A), 
and thus 
B 
(38) |f fl<M@-A). 


Now suppose |x —c| < 6 = 37. Using (37) and then (38) with A =c, B = x, we get 


ig € 
oa = < = —)=e. 
F@)-FOl=|f fs Mle-a <M (z,) =e 
b) Suppose f is continuous at c. We wish to compute 
F'(x) = lim Ae PKG 


xr—->C LS Ee 


Since f is continuous at c, for all « > 0, there exists 6 > 0 such that |x—c| <6 => 
| f(a) — f(e)| < €, or equivalently 

fo) -—e< f(a) < flo +e. 
Therefore 
Eilon It Eiote_ 


CHE SE Ce Z-C 


fle) 


f(o) +¢, 
and thus 

ee 

Cc 

oo <e. 

F — Flo) <e 
This shows that F’(c) exists and is equal to f(c). 
c) By part b), if f is continuous, F(a) = ie f is an antiderivative of f. But we have 
shown that if antiderivatives exist at all they are unique up to an additive constant. 


We have just found an antiderivative F, so if F is any other antiderivative of f we 
must have F(x) = F(x) + C for some constant C’, and then 


b a b 
F() - Fa) = (F) +C) ~ (Fla) +0) = FW) - Fla) = f ¢- f t= fF. 


Remark: Although we introduced the integral “axiomatically”, as long as we are 
only trying to integrate continuous functions we had no choice: the only way to 
assign a value if f to each continuous function f : [a,b] > R satisfying the (rea- 
sonable!) axioms (I1) through (13) is to take peg to be an antidervative F of f 
with F(a) = 0, and again, there is at most one such function. 


These same considerations answer the conundrum of why the celebrated Theo- 
rem 8.1 has such a short and simple proof.! The theorem assumes that we already 


lThis is not just florid language. I taught second semester calculus four times as a graduate 
student and really did become puzzled at how easy it was to prove the Fundamental Theorem 
of Calculus so soon after integration is discussed. I worked out the answer while teaching an 
undergraduate real analysis course at McGill University in 2005. The current presentation is an 
adaptation of my lecture notes from this older course. Soon after I gave my 2005 lectures I found 
that a very similar “axiomatic” treatment of the integral was given by the eminent mathematician 
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have an integral, i.e., an assignment (f : [a,b] + R) ih f for every continuous 
function f. We have shown that there is at most one such integral on the con- 
tinuous functions, but we have not yet constructed this integral! In other words, 
we have settled the problem of uniqueness of the definite integral but (thus far) 
assumed a solution to the much harder problem of existence of the definite integral. 
And again, this existence problem is equivalent to an existence problem that we 
mentioned before, namely that every continuous function has an antiderivative. 

Thus: if we could prove by some other means that every continuous function f 
is the derivative of some other function F’, then by the above we may simply define 
if f = F(b) — F(a). This is the approach that Newton himself took, although 
he didn’t prove that every continuous function was a derivative but rather merely 
assumed it. It is also what freshman calculus students seem to think is taught in 
freshman calculus, namely that the definition of . f is F(b) — F(a)? 

But I do not know any way to prove that an arbitrary continuous function has 
an antiderivative except to give a constructive definition of ie f as a limit of sums 
and then appeal to Theorem 8.1b) to get that ibs f is an antiderivative of f. 


Thus Theorem 8.1 is “easy” because it diverts the hard work elsewhere: we need to 
give a constructive definition of the definite integral via a (new) kind of limiting pro- 
cess and then show “from scratch” that applied to every continuous f : [a,b] > R 


this limiting process converges and results in a well-defined number te of 


2. Building the Definite Integral 
2.1. Upper and Lower Sums. 


Now we begin the proof of the hard fact lurking underneath the Fundamental The- 
orem of Calculus: that we may define for every continuous function f : [a,b] > R 
a number { f so as to satisfy (11) through (13) above. For now, we will make 
a simplifying assumption on our class of integrable functions: namely, let us only 
consider functions f : [a,b] + R such that for every closed subinterval [c, d] C [a, b], 
f : [c,d) > R has a maximum and minimum value. Of course this holds for all 
continuous functions, so it will be a good start. 


The basic idea is familiar from freshman calculus: we wish to subdivide our in- 
terval [a,b] into a bunch of closed subintervals meeting only at the endpoints, and 
then we want to consider the lower sum and upper sum associated to f on each 
subinterval. Then the lower sum should be less than or equal to the “true area 
under the curve” which should be less than or equal to the upper sum, and by 
dividing [a,b] into more and smaller subintervals we should get better and better 
approximations to the “true area under the curve”, so we should define th f via 
some limiting process involving lower sums and upper sums. 


Okay, let’s do it! 


Serge Lang in [L]. So the presentation that I give here is not being given by me for the first time 
and was not originated by me...but nevertheless the material is rarely presented this way. 

2This is not what the books actually say, but what they actually say they don’t say loudly 
enough in order for the point to really stick. 
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Step 1: We need the notion of a partition of an interval [a, b]: we choose finitely 
many “sample points” in [a,b] and use them to divide [a,b] into subintervals. For- 
mally, a partitition P is given by a positive integer n and real numbers 


a=da9 <ay <... S An_1 < Gn =D. 


That is, we require the “first sample point” ao to be the left endpoint of the interval, 
the “last sample point” a, to be the right endpoint of the interval, and the other 
(distinct) points are absolutely arbitrary but written in increasing order. 


Let f : [a,b] > R be any function admitting a minimum and maximum value on 
every closed subinterval of [a,b] (e.g. any continuous function!). For 0 <i<n-—-1, 
let m;(f) denote the minimum value of f on the subinterval [x;, 7;+1] and let M;(f) 
denote the maximum value of f on the subinterval [x;,7;41]. Then we define the 
lower sum associated to f : [a,b] > R and the partition P = {zo,...,¢n} as 


n-1 
L(f,P) = © mi(f)(ein1 — 24) 
i=0 
and also the upper sum associated to f and P as 


UEP) = So MU/)lwina — 2). 
i=0 


These sums have a simple and important geometric interpretation: for any 0 <7 < 
n—1, the quantity x;+1 — 2; is simply the length of the subinterval [x;, 2:41]. So 
consider the constant function m;(f) on the interval [x;, 7:41]: by definition of mj, 
this is the largest constant function whose graph lies on or below the graph of f 
at every point of [x%;,2;4:]. Therefore the quantity m:(f)(xi+1 — v;) is simply the 
area of the rectangle with height m;(f) and width x;41 — 2;, or equivalently the 
area under the constant function y = m;(f) on [a;, v;41]. 


We say the function f : [a,b] > R is integrable if there is a unique I € R 
such that for every partition P of [a,b] we have 


L(f,P) <1 <U(f,P). 


This definition, although correct, is not ideally formulated: it underplays the most 
important part — the uniqueness of I — while making it annoying to show the ezis- 
tence of I. (It turn outs that there is always at least one I lying between every lower 
sum and every upper sum, but this is as yet far from clear.) Here are some examples. 


Example 2.1: If f(a) = C is a constant function, then for every partition P on 
[a,b] we have L(f,P) = U(f,P) = C(b— a). Thus the unique J in question is 
celarly C(b — a): constant functions are integrable. 


Example 2.2: Suppose f(a) is constantly equal to 1 on the interval [a,b] except 
for one interior point c, at which f(c) = 0. We claim that despite having a dis- 
continuity at c, f is integrable, with fe f =b-a. To see this, first observe that 
for any partition P of [a,b] we have U(f,P) = b— a. Indeed this is because on 
every subinterval of [a,b] f has 1 as its maximum value. On the other hand, for 
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any sufficiently small € > 0, we may choose a partition in which c occurs in exactly 
one subinterval (i.e., c is not one of the points of the partition). Then the lower 
sum on that subinterval is 0, whereas on every other subinterval the minimum is 
again 1, so L(f,P) = (b—a)(1—€). This shows that the unique number between 


every L(f,P) and U(f,P) is b—a, so [LPf= (b—a). 


Exercise 2.3: Show that starting with the constant function C on [a, b] and changing 


its value at finitely many points yields an integrable function f with nh f =C(b-a). 


The previous examples have the property that the upper sums U(f,P) are con- 
stant. When this happens, one can show f is integrable by finding a sequence of 
partitions for which the lower sums approach this common value U(f,P) which 
must then be the integral. But constancy of upper sums only occurs in trivial ex- 
amples. For instance, suppose we want to show that f(x) = = is integrable on (0, 1]. 
If we partition [0,1] into n equally spaced subintervals — let us call this partition 
P,, — then since f is increasing its minimum on each subinterval occurs at the left 
endpoint and its maximum on each subinterval occurs at the right endpoint. Thus 


uP=> (2) 2a tS ind (Oo) od 


i=0 i=0 
and 
n-1 ; n 
i+1\ 1 1 . 1 fn(n+1) 1 
UP, Pa) = ‘= 7 =1 
(f, Pn) a ( n ) te ne eS ane) ( 2 ) ss 2n 
Since limy-356 4 = line 5 fol = 3 the upper and lower sums can both be 


made arbitrarily close to $ by taking n to be sufficiently large. Thus if f(x) = x 
is integrable on [0, 1], its integral must be $. Unfortunately we have not yet shown 
that f is integrable according to our definition: to do this we would have to consider 
an arbitrary partition P of [0,1] and show that L(f,P) < 4 < U(f,P). For this 
very simple function f(x) = x it is possible to grind this out directly, but it’s quite 
a bit of work. And that’s just to find the area of a right triangle! 


Example 2.4: Consider the function f : [a,b] — R which is 1 at every irrational 
point and 0 at every rational point. Because every subinterval contains both ratio- 
nal and irrational numbers, for every partition P of [a,b] we have L(f,P) = 0 and 
U(f,P) =1-(b-a) = b—a. Assuming of course that a < b, this shows that f is not 
integrable: rather than the lower sums and the upper sums becoming arbitrarily 
close together, there is an unbridgeable gap between them. 


The previous example perhaps suggests a solution to the problem. It did so to 
the late 19th century mathematician Jean-Gaston Darboux, who came up with an 
elegant way to see whether there is an unbridgeable gap between the lower and 
upper sums. Darboux’s definition crucially and cleverly relies on the existence of 
suprema and infima for subsets of R. In the next section we do things Darboux’s 
way, which as we will see is much more pleasant than our current setup. 


2.2. Darboux Integrability. 
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Let f : [a,b] — R be any function. For a partition P = {a = 1% < 41 <...< 
In-1 <%= b} and 0 <i<n-1, let m;(f) be the infimum of f on [#;,27;41] and 
Mi(f) be the supreumum of f on [x;, 2:41]. Thus we have m;(f) € [—co, co) and 
Mi(f) € (—co, c«]. As above, we define the lower and upper sums associated to P: 


L(f,P) = s mi(f)(xiz1 — i) € [—00, 00), 
i=0 


n-1 
U(f,P) = >) Mi(f)(wi+1 — 2) € (—00, oo]. 
i=0 
For any f and P we have 


(39) L(f,P) < U(f,P). 


Observe though that the lower sum could take the value —oo and the upper sum 
could take the value oo. The following result clarifies when this is the case. 


PROPOSITION 8.2. Let f : [a,b] > R be any function. 
a) The following are equivalent: 
(i) For all partitions P of [a,b], L(f,P) = —co. 
(ii) There exists a partition P of [a,b] such that L(f,P) = —oo. 
(iti) f is not bounded below on [a, 6]. 
b) The following are equivalent: 
(i) For all partitions P of [a,b], U(f,P) = 00. 
(ti) There exists a partition P of [a,b] such that U(f,P) =o. 
(itt) f is not bounded above on {a, b]. 
c) The following are equivalent: 
(i) For all partitions P of [a,b], L(f,P) > —co and U(f,P) < x. 
(it) f is bounded on [a, b]. 


PROOF. a) (i) => (ii) is immediate. 


(ii) = > (iii): We prove the contrapositive: suppose that there is m € R such that 
m < f(x) for all x € [a,b]. Then for all partitions P = {a = 1% <... < Un-1 < 
Ly = b} and all 0 <i <n—1, we have m;(f) >m > —oo, so L(f,P) > —oo. 


(iii) => (i): Suppose f is not bounded below on [a, b], and let P = {a= 29 <...< 
Ln—1 < Ly = b} be a partition of [a,b]. If m;(f) > —oo for allO <i <n-—1, then 
min)og m,(f) is a finite lower bound for f on [a,b], contradicting our assumption. 
So there is at least one i such that m;(f) = —oo, which forces L(f,P) = —oo. 

b) This is similar enough to part a) to be left to the reader. 

c) If for all partitions P, L(f,P) > —co and U(f,P) < ov, then by parts a) and b) 
f is bounded above and below on [a,b], so is bounded on [a,b]. Conversely, if f is 
bounded on [a, 6] then it is bounded above and below on [a,b], so by parts a) and 
b), for all partitions P we have L(f,P) > —oo and U(f,P) < oo. 


Let P; and P2 be two partitions of [a,b]. We say that Pz refines P, if Pz contains 
every point of P,: i.e., if Py C Po. 


LEMMA 8.3. (Refinement Lemma) Let P; C P2 be partitions of [a,b] (i.e., Po 
refines P,). Then for any bounded function f : [a,b] + R we have 


L(f,P1) < L(f, P2) < U(f, P2) < U(f,P1). 
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PROOF. If P2 refines P1, Po is obtained from P, by a finite number of instances 
of the following: choose a subinterval [c,d] of P; and insert a new point e to 
subdivide it into two subintervals. So it is enough to address the case of a single 
interval. Let m and M be the infiumum and supremum of f on [c,d]. Futher, let 
my, and M, be the infimum and supremum of f on [c,e], and mz and My be the 
infimum and supremum of f on [e,d]. Then m = inf m1, mz and M = sup M,, Mo, 
so 

mom, m<mo, M,<M, My <M. 
Then 
m(d—c) = m(e—c) + m(d—e) < mi(e — c) + ma(e — c) 
and 
M(d—c) = M(e-—c)+ M(d—-e) > Mi(e—c) + Mo(e —-c). 


This shows that subdivision causes the lower sum to stay the same or increase and 
the upper sum to stay the same or decrease. Since this holds each time we add a 


point to get from P; to Pz, we get L(f,P1) < L(f,P2) < U(f, Pe) < Lf, Pe). 


LEMMA 8.4. Let f : [a,b] > R be a function, and let P,P be partitions of 
[a,b]. Then L(f,P1) < U(f,P2). That is, any lower sum associated to any partition 
is less than or equal to the upper sum associated to any other partition. 


PROOF. The idea here is simple but important: we choose a common refine- 
ment of P; and P, i.e., a partition which refines (contains) both P; and Pz. Any 
two partitions have infinitely many common refinements, but the most economical 
choice is simply the union of the two: put P = P,; UP2. Then by Lemma 8.3 we 
have 


Now we come to the crux of Darboux’s theory of integrability: for every function 
f : [a,b] + R we define the lower integral int? f as the supremum of L(f,P) as 


P ranges over all partitions of [a, b] and the upper integral ise f as the infimum of 
U(f,P) as P ranges over all partitions of [a,b]. Finally, we say that f is Darboux 
integrable if iy = Ce € R, and we denote this common value by i f. 


LEMMA 8.5. For any function f : [a,b] > R, we have 


[is fs 


PrRoorF. Recall that if X,Y Cc R are such that « < y for all x € X and all 
y € Y, then sup X < inf Y. Now, by Lemma 8.4, for any partitions P; and P2 we 
have L(f,Pi) < U(f,P2). Therefore 


b 


b 
[ faswupa sigur = fF 


a 
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PROPOSITION 8.6. Let f : [a,b] > R be any function. 
a) We have ie = -—o0 iff f is unbounded below. 


b) We have or = 00 iff f is unbounded above. 


c) Therefore (he 2 {ef € Riff f is bounded. In particular, if f is Darboux integrable, 
then it is bounded. 

PRooF. a) If f is unbounded below, iia = supp L(f,P) = supp —00 = —oo. 
If f is bounded below, then for all P, L(f,P) € R and thus supp L(f, P) > —o0. 
b) This is very similar to part a) and is left to the reader. 
c) This follows immediately from parts a) and b). 


In particular the class of Darboux integrable functions satisfies axiom (IOb). 
<b 
Exercise 2.5: Is there a function f : [a,b] > R with ii = De ec oe) 


Finally, here is the result we really want. 


THEOREM 8.7. (Darbouax’s Integrability Criterion) 
For a bounded function f : [a,b] > R, the following are equivalent: 
(i) f is Darboux integrable. 
(ii) For alle > 0 there exists a partition P of [a,b] such that U(f,P) —L(f,P) <e. 
(itt) There is exactly one real number I such that for all partitions P of [a,b], we 


have L(f,P) <1 < U(f,P). 
PROooF. (i) => (ii): Fix e > 0. Since fr = Ls = supp L(f,P), there exists 


—b 
P, with L(f,Pi) > f? —§. Similarly, since [? f = [, f =infp U(f,P), there exists 
P2 with U(f,P2) < ie +§. Let P = P, UP. Since P is a common refinement of 
P, and P2 we have 


b 
(10) USP) <UhPe) < ffs 
and also 
b 
L,P)> LP) > f 1S, 
and thus 
b 
€ 
(41) -LEP)<5- ff. 
Adding (40) and (41) gives 
U(f,P) - L(f,P) <€. 
(ii) ==> (i): By assumption, for every € > 0, there exists a partition P such that 
U(f,P)-L(f,P) < ¢. But by definition, we have Lt <U(f,P) and hig > L(f,P) 
and thus also fr < -L(f,P). Adding these two inequalities gives 


[s : ‘; s <U(f,P) - L(f,P) <e. 
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<b 
Since this holds for all « > 0, we have hag < (oR On the other hand, by Lemma 
=: ee: * 
8.5 we have (eg < oid so ia: = ud € R and thus f is Darboux integrable. 


—b 
(i) = > (iii): Suppose f is Darboux integrable, so i. - thie i =f f €R. Then for 
all partitions P we have ~ 


L(f,P) <fs= fis <fs SUT, P): 


Moreover, suppose I < i f=f 2 f. Then J is less than the supremum of the lower 
sums, so there exists a partition P with I < L(f,P). Similarly, if J > fe f= (ee 
then J is greater than the infimum of the upper sums, so there exists a partition 
P with U(f,P) < I. This shows that ahs f is the unique real number which lies in 
between every lower sum and every upper sum. 

(iii) = > (i): We prove the contrapositive. Suppose that f is not Darboux inte- 
grable. Then for any partition P we have 

—b 


uns fr<f s<0uP) 


<b 
and thus every I € [f : ff ad] lies between every upper sum and every lower sum. 
Ya 


2.3. Verification of the Axioms. 


Let R([a, b]) denote the set of Darboux integrable functions on [a,b]. We now 
tie together the work of the previous two sections by showing that the assignment 
f € Ria, b]) a satisfies the axioms (10) through (13) introduced in §1. In 
particular, this shores up the foundations of the Fundamental Theorem of Calculus 
and completes the proof that every continuous f : [a,b] > R has an antiderivative. 


In summary, we wish to prove the following result. 


THEOREM 8.8. (Main Theorem on Integration) 
a) Every continuous function f : [a,b] > R is Darbousx integrable. 
b) The operation which assigns to every Darbouz integrable function f : [a,b] > R 
the number ce satisfies axioms (10) through (13) above. 
c) Thus the Fundamental Theorem of Calculus holds for the Darboux integral. In 
particular, for every continuous function f, F(x) = i f is an antiderivative of f. 


PRooF. a) Let f : [a,b] — R be continuous. The key is that f is uniformly 
continuous, so for all « > 0, there is 6 > 0 such that for all x1,x2 € [a,Dd], 
|z1 — 2] <6 => |f(x1) — f(x2)| < p£5- Let n € Zt be such that 2=* < 6, and 
let Pn be the partition of [a,b] into n subintervals of equal length boa. Then 
(42) 


U(S.Pa)—Lf,Pa) = (Mf) milf) (“—*) < (= *) SMS) milf). 


1=0 
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Now for allO <i<n-1, m(f) = f(c;) and M;(f) = f(d;) for some c;,d; € 
[vi, Vi41]- Thus \ci = d;| < Ci41 —-%= be — é, sO 


(43) [Mi(f) — ma(f)| = lf (di) — Flea) < b 
Combining (42) and (43) gives 


U(F.Pa) ~ L,Pa) < (2=*) Sans) mis) < (24) ¥ = 


1=0 


—a 


b) (10): By part a), every continuous function f : [a,b] + R is Darboux integrable. 
By Proposition 8.6, every Darboux integrable function on [a,b] is bounded. (I1): 
In Example 2.1, we showed that the constant function C is integrable on [a, b] with 
a C=C(b—a). (12): If fi, fo : [a,b] > R are both Darboux integrable and such 
that fi(z) < fo(x) for all x € [a,b], then for every partition P of [a,b] we have 
L(fi,P) < L(fe,P), and thus 


[ f= spit Pisa FX f2,P y= fa 


(13): Let f : [a,b] — R, and let c € (a,b). Suppose first that f : [a,b] > R is 
Darboux integrable: thus, for all « > 0, there exists a partition P of [a,b] with 
U(f,P) — L(f,P) <«. Let P. = PU {c}. By the Refinement Lemma, 


L(f,P) < Lf, Pe) < U(f, Pe) < UF, P), 


so U(f,P.) — L(f, Pe) < U(f,P) — L(f,P) < ¢«. Let Pi = P.M [a,c] and Pp = 
P.O [e, b]. Then 


Lf, Pe) = L(f, Pi) + Lf, P2), Uf, Pc) = U(f, P1) + U(f, P2), 
and therefore 
(U(f, Pi)-L(F, Pi))+(U(F, Po) -L(Ff, Pi)) = (UF, Pi) +U (Ff, Po)) (LF, Pa) +L, P2)) 
= U(f,P.) — L(f, Pe.) <€ 


so by Darboux’s criterion f : [a,c] + R and f : [c,6] — R are Darboux integrable. 
Conversely, suppose f : [a,c] > R and f : [c,b] > R are Darboux integrable; let 
€ > 0. By Darboux’s criterion, there is a partition P, of [a,c] such that 


U(f,P1) — L(f, Po) < 5 
and a partition P2 of [c, b] such that 
U(f,P2) — Lf, P2) < 
Then P = P, U Py is a partition of [a,b], and 
UEP) — E(F,P) = UE Pa) + UC, Pa) — (LUE, Pa) + LUE, Pa)) 
= (U(F,Pa) - LF, Pa)) + UF, Pa) - LF, Pa) < 5 +5 = 6 


As for the value of the integral: fix « > 0. Let P be any Rae of - bl, Pe = 
P U{c}, Pi = Pe [a,c], P2 = P.M [e, b]. Then 


€ 
5) i 


Cc b 
L(f,P) < Lf, Pe) = Lf, Pi) + L(f, Pa) < | pe / f <U(f,P1) +U(f, Pa) 
=U(f,P-) < U(f,P). 
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Thus fe ft fe f is a real number lying in between L(f,P) and U(f,P) for every 


partition P of [a,b], so by Theorem 8.7 ek + or = ih f. 
c) This is immediate from Theorem 8.1 (the Fundamental Theorem of Calculus!). 


2.4. An Inductive Proof of the Integrability of Continuous Func- 
tions. 
In this section we will give a proof of the Darboux integrability of an arbitrary 
continuous function f : [a,b] + R which avoids the rather technical Uniform Con- 
tinuity Theorem. We should say that we got the idea for doing this from Spivak’s 
text, which first proves the integrability using uniform continuity as we did above 
and then later goes back to give a direct proof. 


THEOREM 8.9. Let f : [a,b] > R be a continuous function on a closed bounded 
interval. Then f is Darbouzx integrable. 


Proor. By Darboux’s Criterion, it suffices to show that for all « > 0, there is 
a partition P of [a,b] such that U(f,P) — L(f,P) <. It is convenient to prove the 
following slightly different (but logically equivalent!) statement: for every « > 0, 
there exists a partion P of [a, b] such that U(f,P) — L(f,P) < (b—a)e. 

Fix € > 0, and let S(e€) be the set of x € [a,b] such that there exists a partition 
P, of [a,b] with U(f,P.) — L(f, Px) < ¢. We want to show b € S(e); our strategy 
will be to show S(e) = [a,b] by Real Induction. 
(RI1) The only partition of [a,a] is P, = {a}, and for this partition we have 
U(f, Pa) = L(f, Pa) = f(a) -0=0, so U(f, Pa) — L(f, Pa) =O <e. 
(RI2) Suppose that for x € [a,b) we have |[a,z] C S(e). We must show that there 
is 6 > 0 such that [a,x + 6] C S(e), and by the above observation it is enough 
to find 6 > 0 such that «+ 6 € S(e): we must find a partition Pz+5 of [a,x + 4] 
such that U(f,Px+s5) — L(f,Pr+s) < (+ 6—-a)e). Since x € S(e), there is a 
partition P, of [a,z] with U(f,P.) — L(f,P2) < («—a)e. Since f is continuous 
at x, we can make the difference between the maximum value and the minimum 
value of f as small as we want by taking a sufficiently small interval around z: i.e., 
there is 6 > 0 such that max(f, [x,x + 6]) — min(f, [z,x + 6]) < €. Now take the 
smallest partition of [x,x+ 6], namely P’ = {x,x+6}. Then U(f,P’) —L(f,P’) = 
(c+6—«)(max(f, |v, x+06])—min(f, [z,2+04])) < de. Thus if we put Pz45 =P, +P’ 
and use the fact that upper / lower sums add when split into subintervals, we have 

U(f, Puts) i Lf, Px+5) = U(f, Px) + U(f, P') = Lif, Px) _ Lif, P’) 

= U(f,P.) — L(f,Px) + U(f,P’) — Lf, P’) < (@ — ae + be = (1 + 6 — ade. 
(RI3) Suppose that for x € (a, b] we have [a, x) C S(e). We must show that x € S(e). 
The argument for this is the same as for (RI2) except we use the interval [x — 6, x] 
instead of [x, 2+]. Indeed: since f is continuous at x, there exists 6 > 0 such that 
max(f, [z—4, z])—min(f, [z—0d, z]) < €. Since r—6 < x, x—6 € S(e) and thus there 
exists a partition P,_5 of [a, z—6] such that U(f, P25) = L(f, Pers) = (c—d—a)e. 
Let P’ = {x — 6, x} and let P; = Py-5 UP’. Then 

U(f, Pr) - Lf, Pr) = U(f, Pr-s) a U(f, P’) =~. (L(f, Pes) als L(f, P’)) 


= (U(f, Px—s) ame Lf, Px—5)) =P d(max(f, [x = }, z}) a min(f, [x % 6, x])) 
< («—6—a)e+ de = (4 — ae. 
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Exercise 2.6: Show that if « € S(e) and a < y < g, then also y € S(e). 


Spivak [S, pp. 292-293] gives a different uniform continuity-free proof of Theo- 
rem 8.9: he establishes equality of the upper and lower integrals by differentiation. 
This sort of proof goes back at least to M.J. Norris [No52]. 


3. Further Results on Integration 
3.1. The oscillation. 


Let f : DC R-R, and let J be an interval contained in the domain D of f. 
We define the oscillation of f on I as 


w(f, 2) = sup(f, 2) — inf(f, Z). 


Note that w(f, J) is in general an extended real number; it is an honest real number 
iff f is bounded on J (which will almost always be the case for us). 


If J CIC D, then inf(f, J) > inf(f,) and sup(f, J) < sup(f, J), and thus 


(44) w(f, J) <w(f, 2). 


Suppose now that c is a point in the interior of the domain D of f. We define the 
oscillation of f at c to be 


w(f, c) a pata w(f, [e = d,e+ 6]). 


In other words, we are considering the oscillation of f on smaller and smaller 
intervals centered around c and taking the limit as 6 approaches zero. Because of 
(44) the function 6 ++ w(f, [c—6,c+]) is an increasing function of 0, so the limit as 
6 approaches zero from the right exists as an element of [0,00] and is simply equal 
to the infimum. What’s the point? This: 


PROPOSITION 8.10. Let I be an interval, f : I — R be a function, and c an 
interior point of I. The following are equivalent: 
(i) w(f,¢) = 0. 


(ii) f is continuous at c. 


PROooF. (i) => (ii): If w(f,c) =0, then for all € > 0, there exists 6 > 0 such 
that w(f,[c — 6,¢ + 6]) = sup(f, [c — 6,¢ + 6]) — inf(f, [ce — 6,c + 4]) < €, ie., there 
exists 6 > 0 such that for all x with |z—c| < 6, |f(x)— f(o)| < €. So f is continuous 
at c. 

(ii) == (i): This is almost exactly the same. If f is continuous at c, then for all 
€ > O there exists 5 > 0 such that for all x with |z—c| < 6, |f(x)—f(c)| < €, and then 
sup(f, [c—6,c+4]) < f(c)+e, inf(f, [e—6,c+4]) > f(c)—e, so w(f, [e—6,c+4]) < 2¢, 
so w(f,c) = lims_,9+ w(f, [c — 6,¢ + 6]) = 0. 


Remark: If f : J > R and c is an endpoint of J, we can still define the oscillation 
w(f,c), just by taking suitable half-intervals: e.g. if c is the left endpoint we put 
w(f,c) =lims_,9+ w(f, [c,¢ + 6]). With this definition and our usual standard con- 
ventions about continuity at an endpoint of an interval, Proposition 8.10 remains 
true even if c is an endpoint of the interval J. 
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Now let f : [a,b] — R and let P = {a = tp < a < ... < &p_1 < Lp = 6} 
be a partition of [a,b]. We define? 

n-1 

(LP) = So w(F, [vis vial) (aig — es) = UCf,P) - Lf, P). 

i=0 
Thus this notation is just a way of abbreviating the quantities “upper sum minus 
lower sum” which will appear ubiquitously in the near future. We can restate Dar- 
boux’s Criterion especially cleanly with this new notation: a function f : [a,b] > R 
is integrable iff for all « > 0, there exists a partition P of [a,b] with A(f,P) <. 


3.2. Discontinuities of Darboux Integrable Functions. 


At this point, I want to discuss the result that a bounded function f : [a,b] > R 
with only finitely many discontinuities is Darboux integrable. So I wrote up a “di- 
rect” proof of this and it was long and messy. Afterwards I realized that a better 
argument is by induction on the number of discontinuities. One then has to prove 
the result for a function with a single discontinuity (base case), and assuming the 
result for every function with n discontinuities, prove it for every function with 
n+ 1 discontinutities (inductive step). Here the inductive step is especially easy: 
if f : [a,b] > R has n +1 points of discontinuity, we can choose c € (a,b) such 
that f|{a,q has exactly one discontinuity and f\|j-,yj has exactly n discontinuities. 
The restricted functions are Darboux integrable by the base case and the induction 
hypothesis, and as we know, this implies that f : [a,b] > R is Darboux integrable. 

So really it is enough to treat the case of a bounded function with a single 
discontinuity. It turns out that it is no trouble to prove a stronger version of this. 


THEOREM 8.11. Let f : [a,b] > R be bounded. Suppose that for all c € (a,b), 
F\fc,o + [c,b] + R is Darboua integrable. Then f is Darboux integrable and 


tin fre fs 


Proor. Let M > 0 be such that |f(a)| < © for all 2 € [a,b]. Fix 6 > 0 and 
consider partitions P of [a,b] with cz} = a+. For such partitions, 


Af, P) = AF, PO [a,a+ 6]) + A(f,PA [a + 5,0)). 


Since the infimum of f on any subinterval of [a, 6] is at least —M and the supremum 
is at most M, A(f,[a,a+ 6]) < 2M6, which we can make as small as we wish by 
taking 6 small enough. Similarly, having chosen 6, we may make A(f,PM [a+ 6, d]) 
as small as we like with a suitable choice of P, since f is assumed to be Darboux 
integrable on [a + 6,6]. Thus we can make the oscillation at most ¢ for any € > 0, 
so f is Darboux integrable on [a,b]. The second statement follows easily: 


fs Co ee 


and the last quantity goes to zero as c > a* 


3For once we do not introduce a name but only a piece of notation. In an earlier course on 
this subject I called this quantity “the oscillation of f on P”, but this is not especially apt. Better 
perhaps would be to call A(f,P) the discrepancy of f and P, since it is the difference between 
the upper and the lower sum. But in fact it is simplest not to call it anything but A(f,P)! 
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Of course there is an analogous result with the roles of a and 6 reversed. 


This result is also telling us that under certain situations we need not bother to 
consider “improper integrals”: the improper integral will exist iff the conventional 
Darboux integral exists. This will make a lot more sense in the context of a discus- 
sion on improper integrals, so we defer the point until then. 


THEOREM 8.12. Let f : [a,b] > R be a bounded function which is continuous 
except at a finite set of points in its domain. Then f is Darboux integrable. 


Exercise 3.1: Use Theorem 8.11 (and its reflected version) to prove Corollary 8.12. 
THEOREM 8.13. A monotone function f : [a,b] > R is Darboux integrable. 


PrRooF. By reflection it suffices to deal with the case of a weakly increasing 
case. For such functions a miracle occurs: for every partition P = {a = 2% < x < 
<< In-1 < Ln = b} of [a,b] and for all 0 < i < n—1, the infimum of f on 
[vi, %i41] is attained at the left endpoint x; and the supremum of f on [x;, 7:41] is 
attained at the right endpoint x7;,,. Therefore 
L(f,P) = f(wo)(t1 — to) + f(a1)(@2 — 1) +... + f(@n—-1)(@n — Pn~1); 
U(f,P) = f(x1)(@1 — £0) + f(v2)(@2 — 1) +... + f (Gn) (Gn — Bn—1). 
Things simplify further if we simply take P,, to be the uniform partition of [a, }] 
into n equal parts. Then 
b-a 
L(f,Pn) = n (f(xo) reset f(@n-1)) ; 


UE, Pr) =" —9 (Fan) +--+ Sen): 


so 


UUPa) ~ E(F,Pn) = (2=*) (10) ~ Fla). 


Thus taking n sufficiently large we may take U(f, Pn) — L(f, Pn) arbitrarily small, 
so f is Darboux integrable by Darboux’s criterion. 


Theorem 8.13 goes beyond Theorem 8.12: there are increasing functions which are 
discontinuous at infinitely many points. To construct such things becomes much 
easier with some knowledge of infinite sequences and series, so we defer this discus- 
sion until later, except to make the following advertisement: for any real numbers 
a <b and any injective function s : Zt — [a,b], there exists an increasing function 
f : [a,b] + R which is discontinuous at precisely the points s(n) € [a, }]. 


Example 3.2 (Thomae’s Function): Let f : [0,1] — R be defined by f(0) = 0, 
f (7) = a and f of any irrational number is zero. Then f is continuous at 0 and at 
all irrational numbers but discontinuous at every rational number. Thus not only 
does f have infinitely many points of discontinuity in [0,1], they are dense: any 
nontrivial subinterval contains at least one point of discontinuity. We claim that 
nevertheless f is Darboux integrable and i f =0. First observe that since every 
subinterval [x;,7;41] contains an irrational number, the infimum of f on [x;, 7:41] 
is zero, so for any partition P the lower sum is L(f,P) = 0. It follows then that 
is = supp L(f,P) = 0, and thus if f integrable, its integral is zero. It remains to 
show that for each « > 0 we may find a partition P of [0,1] such that U(f,P) < e. 
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To see this, observe that for any fixed e, there are only finitely many nonzero ratio- 
nal numbers 7 in [0,1] with q > e: indeed there is at most 1 such with denominator 
1, at most 2 with denominator 2, and so forth (and in fact there are less than 
this because e.g. in our terminology the “denominator” of $ is actually 2, since 
? = 4 in lowest terms). Suppose then that there are N points z in (0, 1] such that 
f(x) >. Choose a partition P such that each of these points x lies in the interior 
of a subinterval of length at most >. Since the maximum value of f on [0,1] is 1, 
the term of the upper sum corresponding to each of these N “bad” subintervals is 
at most 1 - 557; since there are N bad subintervals over all, this part of the sum 
is at most N - =, and the remaining part of the sum is at most € times the 
length of [a,b] = [0,1], ie., at most e«. Thus U(f,P) > e+ .€ = 2e. Since of course 


lim,_9 2e = 0, this shows that f is Darboux integrable. 


All of our results so far have been in the direction of exhibiting examples of Darboux 
integrable functions with increasingly large sets of discontinuities. What about the 
other direction: is there, for instance, a Darboux integrable function which is dis- 
continuous at every point? In fact, no: 


THEOREM 8.14. Let f : [a,b] > R be Darbousx integrable. Let S' be the set of 
x € [a,b] such that f is continuous at x. Then S is dense in [a,b]: i.e., for all 
a<a<y<b, there exists z € (x,y) such that f is continuous at z. 


PROOF. Step 1: We show that there is at least one c € [a,b] such that f is 
continuous at c. We will construct such ac using the Nested Intervals Theorem: 
recall that if we have a sequence of closed subintervals [a,,, b,] such that for all n, 
ean < bn, 
ean < Qn+1; 
bd Dn+1 < bn, 
there is at least one c such that a, < c < b, for all n: indeed sup, ay < inf, bn, 
so any c € [sup,, @n, inf, by] will do. Since f is Darboux integrable, for all n € Zt 
there is a partition P, = {a= % <@1<...<@n_-1 < Un = 0} of [a,b] such that 

n-1 


(45) ACP, Pn) = )_ wf, (es, vi41]) (Cin. — Bi) < 


i=0 


b-a 
a 


Now (45) implies that for at least one 0 < i <n —1 we have w(f, [x;,2i41]) < +: 
for, if not, w(f, [wi,vi+1]) > + for all i and thus 


1 i In—% b-a 


A(f, Pn) = “(0 Xo) + —(@2 — 21) +... + —(8@n — Ln-1) = ——— = : 


n n n n 


contradiction. We will use this analysis to choose a nested sequence of subintervals. 
First we take n = 1 and see that there is some closed subinterval [x;, 7:41] of [a, 6] 
on which w(f, [w;,2i41]) < 1. We then define ay = 2;, b) = 4:41, and instead of 
considering f as defined on [a,b], we now consider it as defined on the subinterval 
[a,,5,]. Since f is Darboux integrable on [a, b], we know it is also Darboux integrable 
on [a1, 01], so the above argument still works: there exists a partition P2 of [a1, bi] 
such that for at least one subinterval [#;, 2:41] C [ai, bi] we have w(f, [2;, j41]) < 
$. We then put ag = 2x; (this is not necessarily the same number that we were 
calling 2; in the previous step, but we will stick with the simpler notation) and 
bg = xj41 and have defined a sub-subinterval [a2,b2] C [ai,bi] C [a,] on which 
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w(f,[a2,b2]) < 4+. Now, continuing in this way we construct a nested sequence 


[@n, bn] of closed subintervals such that for all n € Z*, w(f,[an,bn]) < +. Now 


apply the Nested Intervals Theorem: there exists c € R such that c € [an, bn] for 
alln € Zt. It follows that for all n € Z* 


w(F,6) Saf [an bo))) <=, 


i.e., w(f,c) = 0 and thus f is continuous at c by Proposition 8.10. 
Step 2: To show f has infinitely many points of continuity, it’s enough to show that 
for all N € Z* f is continuous at at least N distinct points, and we can do this by 
induction, the base case N = 1 being Step 1 above. So suppose we have already 
shown f is continuous at 71 < 42 <...< @y in [a,b]. Choose any A, B € R with 
a<a<A<B<2a2<b. Once again, since f : [a,b] > R is Darboux integrable, 
the restriction of f to [A, B] is Darboux integrable on [A, B]. Applying Step 1, we 
get c € [A, B] such that f is continuous at c, and by construction c is different from 
all the continuity points we have already found. This completes the induction step, 
and thus it follows that f is continuous at infinitely many points of [a, b]. 


3.3. A supplement to the Fundamental Theorem of Calculus. 
THEOREM 8.15. Let f : [a,b] > R be differentiable and suppose f’ is Darboux 
integrable. Then eh f’ = f(o) — fla). 


Proor. Let P be a partition of [a,b]. By the Mean Value Theorem there is 
ty € [vi, v;41] such that f(ai41) — f(ai) = f'(ti) (ina — 21). Then we have 


mi(f’)(tigr — vi) < f (ti) (wig — vi) < Mi(f’) (wisi — 2) 
and thus 

ma(f’)(xit1 — 21) < f(tis1) — fei) S Mi(F') (ti41 — 22). 
Summing these inequalities from i = 0 to n — 1 gives 


Since for the integrable function f, fe f is the unique number lying in between all 


lower and upper sums, we conclude f(b) — f(a) = Ay f'. 


How is Theorem 8.15 different from Theorem 8.1c)? Only in a rather subtle way: 
in order to apply Theorem 8.1c) to f’, we need f’ to be continuous, whereas in 
Theorem 8.15 we are assuming only that f’ is Darboux integrable. Every contin- 
uous function is Darboux integrable but, as we have seen, there are discontinu- 
ous Darboux integrable functions. What about discontinuous, Darboux integrable 
derivatives? The possible discontinuities of a monotone function are incompatible 
with the possible discontinuities of a derivative: if f’ is monotone, it is continuous. 
So we must look elsewhere for examples. In fact, we return to an old friend. 


Example 3.3: Let a,b € (0,00) and let fa» be given by t +> x sin(,), xr #0 
and 0++ 0. Then f,,» is infinitely differentiable except possibly at zero. It is con- 
tinuous at 0, the sine of anything is bounded, and lim, 49° = 0, so the product 
approaches zero. To check differentiability at 0, we use the definition: 

f(h) =f) _ |, bh sin( ge) 


1 
a = li ee SS SS pe eS eg a-1le: — 
ar h ae eo ae 
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This limit exists and is 0 iffa—1> 0 iffa>1. Thusifa>1 then f’ ,(0)=0. As 
for continuity of f/ , at zero, we compute the derivative for nonzero x and consider 
the limit as « > 0: 


1 1 
! a-1.: a—b-1 
rv) =ax sin(—-) — ba 3(—). 
fa.ol )=a sin( a cos(-F) 


The first term approaches 0 for a > 1. As for the second term, in order for the 
limit to exist we need a > 6+ 1. This calculation shows that f’ , is continuous at 
0 iff a > b+ 1; so in this case we can apply the first version of the Fundamental 
Theorem of Calculus to conclude for instance that 


i P= Fa@iahai— hue: 


Next, ifa <b+1, then f’ , is unbounded near 0, hence is not Darboux integrable 
on any interval containing zero. But there is a third case: if a = b+ 1, then 
iMz-+0 ie does not exist, but ae is bounded on any closed, bounded interval, say 
0,2]. Therefore Theorem 8.15 applies to give 


i Sosa = fo+ie(©) — fo41,0(0) = fori6(2) 


for all b> 0. 


This example, delicate though it is, provides the first evidence that the Darboux 
integral may not be the last word on integration theory. It is natural to want a 
fundamental theorem of calculus in which no hypothesis is needed on f’. Thus we 
want an integration theory in which every derivative f’ is integrable. As we have 
just seen, the Darboux integral is not such a theory. In graduate real analysis, 
one meets a more powerful and general integral, the Lebesgue integral, which 
remedies many of the defects of the Darboux integral...but not this one! In fact for 
b >a+1 the derivatives Fis are not Lebesgue integrable either. 


3.4. New Integrable Functions From Old. 


In this section we show that performing many familiar, elementary operations on 
integrable functions yields integrable functions. By showing any interest in these 
results we are exploring the extent that the class of Darboux integrable functions 
goes beyond the class of all continuous functions on [a,b]. Indeed, we know that 
every continuous function is integrable, and all of the operations we are discussing 
here will take continuous functions to continuous functions. 


THEOREM 8.16. Let f,g : [a,b] > R be Darbousx integrable functions. 


a) For any constant C, Cf is Darboux integrable and fe Cf= on f. 
b) The function f + g is Darbouz integrable, and moreover 


[sto= fos fis 


PROOF. a) The idea here is simply that C may be factored out of the lower 
sums and the upper sums. The details may be safely left to the reader. 
b) Let J C [a,b] be a subinterval, and let m;,m,,my4, be the infima of f, g 
and f +g, respectively on J. Things would certainly be easy for us if we had 
mr + mg = Mfg, but observe that this need not be the case: e.g. consider 
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f(x) = and g(x) = —a on [—1,1]. Then my = —1, m, = —1 and my4, = 0, so 
my +m, < my4g. However there is a true inequality here: we always have 
mp + Mg << Mf4q- 
Applying this on every subinterval of a partition P gives us 
L(f,P) + L(g,P) < L(f +9,P). 
Similarly, denoting by M;, M, and My +, the suprema of f, g and f + g on some 
subinterval J, we have 
Ms + My = Mf+q 
and this implies that for every partition P of [a,b] we have 
U(f+9,P) <U(f,P) +U(g,P). 
Combining these inequalities gives 
(46) L(f,P) + L(g, P) < L(f +9,P) <U(f+g9,P) <U(f,P)+U(9,P). 
Moreover, subtracting the smallest quantity from the largest gives 
0<A(f+g9,P) < AF,P) +A, P); 
since f and g are Darboux integrable, for ¢ > 0, there is P, such that A(f,P1) < § 
and P2 such that A(g,P2) < 5. Taking P = P; U Pe, both inequalities hold, so 


€ € 
A(f+g,P) SAF, P)+ AGP) <5 +5 =6 
By Darboux’s Criterion, f+g is Darboux integrable, and the inequalities (46) imply 
b b b 
Lfto= fi Peto: 
THEOREM 8.17. Let f : [a,b] — [c,d] be Darbousx integrable and g: [c,d] > R 
be continuous. Then the composite function go f : [a,b] + R is Darboux integrable. 


PROOF. Since g is continuous on [c,d], by the Extreme Value Theorem it is 
bounded: there exists M such that |g(x)| < M for all x € [c,d]. 

Fix « > 0. By the Uniform Continuity Theorem, there exists 7 > 0 such that 
lc —y| <n => |9(x) — 9(y)| < peaham- Shrinking 7 if necessary, we may assume 


€ 
b-a+2M 

Since f is Darboux integrable, there exists a partition P = {a = 1% <a@1<...< 
In—1 < Xn = bd} of [a,b] such that A(f,P) < 77. 

We divide the index set {0,1,...,— 1} into two subsets: let S; be the set of 
such 7 such that w(f, [vi, i41])(@iz1 — vi) <7, and let Sg be the complementary 
set of all 7 such that w(f, [vi, ti41])(ti41 — vi) > 7. We have set things up such 
that for alli € S1, w(gof, [zi, zi41]) < poghaqz- Since Si C {0,...,n—1}, we have 


n< 


n-1 
So (eis —xi)< So (wisn —2a)=b-a. 
1E Sy i=0 


On the other hand, since —M < f(x) < M for all x € [a, }], the oscillation of f on 
any subinterval of [a, b] is at most 2M. Thus we get 


A(go f,P) = >> w(f, [we viti]) (wigs — 1) + SS WS, (wi, tit) (wit — 24) 


4€S1 1ES2 
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€ ye (Gi44 —2i) + (2M) De (Ui41 Xi) < ie IM (b a) L 2M y, (Ui41 — i). 
i€Sy i€So tES2 

(Note that reasoning as above also gives })j¢5, Vit1 — Ti < (b— a), but this is not 

good enough: using it would give us a second term of 2M(b—a), i.e., not something 

that we can make arbitrarily small.) Here is a better estimate: 


do itt — 2) = 2 >) neis1 — 4) < = > (S, (ae, e+) (ei41 — 24) 


i€S2 4€S2 t1€S2 
eee 1 i: & 
in do oF, [ti, ®i41])(@i41, 21) = —A(f,P) < —n° = 7. 
ie n 0 
Using this estimate, we get 
€ (b—a)e 2Me 
A << b + 2M =e. 
Wel) Sz aaom oe eo bag aM 


The proof of Theorem 8.17 becomes much easier if we assume that g is not merely 
continuous but a Lipschitz function. Recall that f : J + R is Lipschitz if there 
exists C' > 0 such that for all x,y € J, |f(x) — f(y)| < Cla — yI. 


Example: f : R > R by x +4 || is a Lipschitz function. Indeed, the reverse 
triangle inequality reads: for all x,y € R, 

Iz] —|yll < lz — yl, 
and this shows that 1 is a Lipschitz constant for f. 


Exercise: a) For which functions may we take C = 0 as a Lipshitz constant? 
b) Let J be an interval. Show that for every Lipshitz function f : J > R, there is 
a smallest Lipschitz constant. 


PROPOSITION 8.18. Let f : [a,b] + R be aC'-function. Then M = maxyejaj |f'(x)| 
is a Lipschitz constant for f. 


Proor. Let x < y € [a,b]. By the Mean Value Theorem, there is z € (a, y) 
such that f(x) — f(y) = fey), 80 [f(e)—Fy)| < LF"@)lle—yl < Mle—ul 


LEMMA 8.19. Let f : I — [c,d] be bounded and g : [c,d] + R a Lipshitz function 
with Lipschitz constant C. Then w(go f,I) < Cw(f,1). 


Exercise: Prove Lemma 8.19. 


THEOREM 8.20. Let f : [a,b] — [c,d] be Darbouz integrable, and let g : [c,d] > 
R be Lipschitz with contant C. Then go f : [a,b] > R is Darboux integrable. 


PRooF. Fix « > 0. Since f is integrable, there is a partition P of [a, b] with 


n-1 
€ 


ACE, P) = D5 oF, [2s 241) (win — 24) < rah 


i=0 
Then by Lemma 8.19 we have A(go f,P) = 39 w(go f, (ws, Vis11) (ina — i) 


<C (Sour [vi, ita]) (Vig — «)] <C (5) =e. 


i=0 
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COROLLARY 8.21. Let f : [a,b] > R be Darboux integrable. Then |f|: [a,b] > 
R is Darboux integrable, and we have the integral triangle inequality 


[ss fit 


PROOF. Since g(x) = |z| is a Lipshitz function, by Theorem 8.20 go f =|f| is 
Darboux integrable on [a,b]. Moreover, since —|f| < f < |f|, by (12) we have 


-fuis firs fia 
[nsf 


COROLLARY 8.22. Let fi, fo : [a,b] > R be Darboux integrable. Then the 
product fi fz : [a,b] > R is Darboux integrable. 


sO 


PrRooF. It is really just a dirty trick: we have the identity 


eee (fi + f2)? ; (fi ani 


Now, by Theorem 8.15, both f; + fo and fi — fo are Darboux integrable. Since f; 
and f2 are Darboux integrable, they are both bounded, so there exists M with 
fi({a, b]), fo([a,b]) C [-M,M]. The function g(x) = x? is Ct on the closed, 
bounded interval [-M, M] and thus Lipschitz there. Thus Theorem 8.20 applies 
to show that (f; + fo)? and (f1; — fa)? are Darboux integrable, and then Theorem 
8.16 appplies again to show that f) fo is Darboux integrable. 


Warning: It is usually not the case that fr fife= ie fi r fe! 


Example 3.6: Let f : [1,2] — [0,1] be the function which takes the value i at 
every rational number a and 0 at every irrational number, and let g : [0,1] ~ R 
be the function which takes 0 to 0 and every x € (0,1] to 1. Then f is Darboux 
integrable, g is bounded and discontinuous only at 0 so is Darboux integrable, but 
go f : [1,2] — R takes every rational number to 0 and every irrational number to 
1, so is not Darboux integrable. Thus we see that the composition of Darboux in- 
tegrable functions need not by Darboux integrable without some further hypothesis. 


Example 3.7: Above we showed that if g is continuous and f is Darboux inte- 
grable then the composition go f is Darboux integrable; then we saw that if g 
and f are both merely Darboux integrable, go f need not be Darboux integrable. 
So what about the other way around: suppose f is continuous and g is Darboux 
integrable; must go f be Darboux integrable? The answer is again no; the easiest 
counterexample I know is contained in a paper of Jitan Lu [Lu99]. 


170 8. INTEGRATION 


4. Riemann Sums, Dicing, and the Riemann Integral 


We now turn to the task of reconciling G. Darboux’s take on the integral with 
B. Riemann’s (earlier) work. Riemann gave an apparently different construction 


of an integral ik f which also satisfies axioms (10) through (13). By virtue of the 
uniqueness of the integral of a continuous function, the Riemann integral R de f of 


a continuous function agrees with our previously constructed Darbouz integral dhe f. 
But this leaves open the question of how the class of “Riemann integrable func- 
tions” compares with the class of “Darboux integrable functions”. In fact, although 
the definitions look different, a function f : [a,b] + R is Riemann integrable iff it 
is Darboux integrable and then R fp f= i f. Thus what we really have is a rival 
construction of the Darboux integral, which is in some respects more complicated 
but also possesses certain advantages. 


It turns out however to be relatively clear that a Riemann integrable function 
is necessarily Darboux integrable. This suggests a slightly different perspective: we 
view “Riemann integrability” as an additional property that we want to show that 
every Darboux integrable function possesses. This seems like a clean way to go: one 
the one hand, it obviates the need for things like R i f. On the other, it highlights 
what is gained by this construction: namely, further insight on the relationship of 
the upper and lower sums U(f,P) and L(f,P) to the integral i f. At the moment 
the theory tells us that if f is Darboux integrable, then for every « > 0 there exists 
some partition P. of [a,b] such that U(f,P.) — L(f,P.) < €. But this is not very 
explicit: how do we go about finding such a P.? In the (few!) examples in which 
we showed integrability from scratch, we saw that we could always take a uniform 
partition P,,: in particular it was enough to chop the interval [a, b] into a sufficiently 
large number of pieces of equal size. In fact, looking back at our first proof of the 
integrability of continuous functions, we see that at least if f is continuous, such 
uniform partitions always suffice. The key claim that we wish to establish in this 
section is that for any integrable function we will have A(f,P,,) < € for sufficiently 
large n. In fact we will show something more general than this: in order to achieve 
A(f,P) < €, we do not need P to have equally spaced subintervals but only to have 
all subintervals of length no larger than some fixed, sufficiently small constant 06. 


Given a function f : [a,b] > R and a partition P of [a,b], we will also introduce 
a more general approximating sum to i f than just the upper and lower sums, 
namely we will define and consider Riemann sums. The additional flexibility of 
Riemann sums is of great importance in the field of numerical integration (i.e., 
the branch of numerical analysis where we quantitatively study the error between 
a numerical approximation to an integral and its true value), and it pays some 
modest theoretical dividends as well. But the Riemann sums are little more than 
a filigree to the main “dicing” property of the Darboux integral alluded to in the 
last paragraph: the Riemann sums will always lie in between the lower and upper 
sums, so if we can prove that the upper and lower sums are good approximations, in 
whatever sense, to the integral fe f, then the same has to be true for the Riemann 
sums: they will be carried along for the ride. 
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4.1. Riemann sums. 


Let f : [a,b] — R be any function, and let P be a partition of [a,b]. Instead 
of forming the rectangle with height the infimum (or supremum) of f on [2;, 7:41], 
we choose any point x* € [x;,%;+41] and take f(z) as the height of the rectan- 
gle. In this way we get a Riemann sum eo f(%7)(ia1 — 2) associated to 
the function f, the partition P, and the choice of a point x* € [x;,2;4,] for all 
0<i<n-1. Given a partition P = {a = % < t1 <... < Un-1 < Ln = Dh, 
a choice of x* € [a;, 2:41] for 0 < i < n—1 is called a tagging of P and gets 
a notation of its own, say T = {xj,...,v%_,}. A pair (P,7) is called a tagged 
partition, and given any function f : [a,b] > R and any tagged partition (P,7) 
of [a,b], we associate the Riemann sum 

n-1 


R(f,P,7) = >> f (27) wisi — 2%). 


i=0 


Let us compare the Riemann sums R(f,7) to the upper and lower sums. Just 
because every value of a function f on a (sub)interval J lies between its infimum 
and its supremum, we have that for any tagging r of P, 


(47) L(f,P) < R(f,P,7) < U(f,P). 


Conversely, if f is bounded then for all « > 0 we can find a7,a7* € [x;, 2:41] 


such that sup(f, [z;, vi41]) < f(vt) + € and inf(f, [z;,vi41]) > f(xi*) — «, and it 
follows that the upper and lower sums associated to f and P are the supremum 
and infimum of the possible Riemann sums R(f,P,7): 


(48) sup R(f,P,7) = U(f,P), inf R(f,P,7) = L(f,P). 

Exercise 4.1: Show that (48) holds even if f is unbounded. More precisely, show: 
a) If f is unbounded above, then sup, R(f,P,7) = U(f,P) =~. 

b) If f is unbounded below, then inf, R(f,P,7) = L(f,P) = —oo. 

From inequalities (47) and (48) the following result follows almost immediately. 


THEOREM 8.23. For a function f : [a,b] > R, the following are equivalent: 
(i) f is Darboux integrable. 
(ii) For alle > 0, there exists a real number I and a partition P. of [a,b] such that 
for any refinement P of P. and any tagging t of P we have 


|IR(f,P,7) —I| <e. 
If the equivalent conditions hold, then I = ie ie 


PRoor. (i) => (ii): If f is Darboux integrable, then by Darboux’s Criterion 
there is a partition P. such that A(f,P.) = U(f,P-) — L(f,Pe) < ¢. For any 
refinement P of P, we have A(f,P) < A(f,P.-), and moreover by integrablity 


b 
LP) < f FSU). 
For any tagging 7 of P we have also 


L(f,P) < R(f,P,7) < U(f,P). 
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Thus both R(f,P,7) and i lie in an interval of length less than ¢, and it follows 


that their distance from each other, |R(f,P,7) — aN f|, is less than e. 
(ii) =— > (i): Fix e > 0. Since U(f,P.) = sup, R(f,Pe,7) and L(f,P.) = 
inf, R(f,P.,7), if |R(f,P.,7) — I] < ¢ for all 7, then |U(f,P.) — I] < € and 
|L(f,P.) — I| < € and thus U(f,P.) — L(f,P.) < 2. Since € was arbitrary, f 
is Darboux integrable. Moreover, the only number J such that for all « > 0, there 
is a partition P. with |U(f,P.) —I| < , |L(f,P.) —1| < € is i fa 


Theorem 8.23 gives a sense in which a function is Darboux integrable iff the Rie- 
mann sums R(f,P,7) “converge” to i f. However, the proof involves little more 
than pushing around already established facts. Riemann considered a different, a 
priori stronger, sense in which the Riemann sums converge to i f. In the next 
section we discuss this and show that it holds for every Darboux integrable function. 


4.2. Dicing. 


For a partition P = {a = 2 < @ <.... < Gn_1 < Ly = 5} of [a,b], its mesh 
|P| is max;(x%:41 — @), ie., the largest length of a subinterval in P. The mesh of 
a partition is a better measure of its “size” than the number of points it contains. 
One can think of a kitchen assistant dicing vegetables — making a lot of knife cuts 
doesn’t ensure a good dicing job: you might have some tiny pieces but also some 
large pieces. Rather a proper dicing will ensure the mesh is sufficiently small. 


LEMMA 8.24. Let P; be a partition of [a,b] and let Pz D Py be a refinement of 
Pi. Then |P2| < |Pi|. 


Proor. Proving this is merely a matter of absorbing the definitions, so we 
leave it to the reader as a good opportunity to stop and think. 


LEMMA 8.25. (Dicing Lemma) Let f : [a,b] + R be bounded. For all € > 0, 
there exists 6 > 0 such that for all partitions P of [a,b] with |P| < 6, 


b b 
(49) [ f-HP) <canduts.P)- ff <e 


Proor. (Levermore) Let ¢ > 0. There exists a partition P, of [a,b] such that 


b 
o< f f-LUP<§, 0<U(f,P pe] 428 


Suppose |f(a)| < M for all x € [a,b]. Let N be the number of subintervals of P.. 
Choose 6 > 0 such that 2MN6 < 5. We claim (49) holds for any partition P with 
|P| < 6. Indeed, let P be any partition with |P| < 6, and put P’ = PUP... Now 


b b 
(50) O< / f-LEP) = ( ‘| f-L(L,P’)) + (LU, P") - LP), 


(51) O<U(f,P )-f'r=w pf’ nsw UF, P’)). 


We will establish the claim by showing that the two terms on the right hand side of 


(50) are each less than § and, similarly, that the two terms on the right hand side 
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of (51) are each less than §. Using the Refinement Lemma (Lemma 8.3), we have 


b b 
o< f f-unP)s f[ t-L,P)< 
and 


Ab Ab 
o<u(pp)- f fsusP)- f f< 


This gives two of the four inequalities. As for the other two: since P’ is a refinement 
of P = {a = 2% <... < ... < @n_-1 < ty = Db}, forO Si < N-1, Pi c= 
PN (xi, vi41] is a partition of [x;, 7:41]. By the Refinement Lemma, 


Nl 


0<L(f,P’)-L(f,P) = yy (L(f, Ps) — inf (f, [zi vi41])) , 


0<U(f,P)—U(f,P’) = >> (sup(f, (wi, ei41]) — UC, Pi). 
i=0 
Because P’ has at most N —1 elements which are not contained in P, there are at 
most N — 1 indices 7 such that (#;,2;41) contains at least one point of P/. For all 
other indices the terms are zero. Further, each nonzero term in either sum satisfies 


0 < L(f, Pi) = inf(f, [vi, Ti41]) < 2M (@i4a = Xi) < 2Mo, 
0 < sup(f, [vi, vi41]) — U(f, Pi) < 2M (xizi — 2:1) < 2M. 
Because there are at most N — 1 nonzero terms, we get 


0< L(f,P’) -L(f,P) < 2MN6 < 


0 <U(f,P) —U(f,P’) < 2MN6 < Bs 


So the last terms on the right hand sides of (50) and (51) are each less than 5. 


We can now deduce the main result of this section. 


THEOREM 8.26. a) For a function f : [a,b] > R, the following are equivalent: 
(i) f ts Darboux integrable. 
(ii) There exists a number I such that for alle > 0, there exists 6 > 0 such that for 
all partitions P of [a,b] of mesh at most 6 and all taggings t of P, 


|IR(f,P,7) —I| <e. 
(iti) For every sequence (Pn, Tm) of tagged partitions of [a,b] such that |P,| > 0, 
the sequence of Riemann sums R(f,Pn,T) 1s convergent. 
b) If condition (it) holds for some real number I, then necessarily I = ie f. 
c) If condition (itt) holds, then for every sequence (Pn, Tn) of tagged partitions with 
[Pn +0, R(f, Pasta) > fo f 


PROOF. a) (i) = > (ii): if f is Darboux integrable, then Sf = (ie? and 
property (ii) follows immediately from the Dicing Lemma (Lemma 8.25). 
(ii) == (iii): Indeed, if (ii) holds then for any sequence of tagged partitions 
(Pn, Tn) with |P,| + 0, we have R(f,Pn,T) > I. 
(iii) — > (i): We will show the contrapositive: if f is not Darboux integrable, 
then there is a sequence (Pn,T) of tagged partitions with |P,,| > 0 such that the 
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sequence of Riemann sums R(f,Pn, Tn) is not convergent. 

Case 1: Suppose f is unbounded. Then for any partition P of [a,b] and any M > 0, 
there exists a tagging 7 such that |R(f,P,7)| > M. Thus we can build a sequence 
of tagged partitions (Pn, 7m) with |Pn| > 0 and |R(f,Pn,tm)| 4 ©. 

Case 2: Suppose f is bounded but not Darboux integrable, i.e., 


-w< [p< [peo 


For n € Zt, let P, be the partition into n subintervals each of length ba Since 
U(f,P) =sup, R(f,P,7) and L(f,P) = inf, L(f,P,7), for all n € Z* there is one 
tagging t, of P, with L(f,Pn) < R(f,Pn,tn) < L(f, Pn) + + and another tagging 
Ty of Py with U(f,Pn) > R(f, Pn, Tn) > Uf, Pn) - 1. By the Dicing Lemma, 


lim L(f.P = fh jim unr = [i 


and it follows, for instance by a squeezing argument, that 


lita R(f, Pa, tn) - [fs 


noo 


lim R(f, Pn, Tn) =f f. 

n—-oo 
Now let 7, be t, if n is odd and T,, is n is even. Then we get a sequence of tagged 
partitions (Pp, T) with |P,,| + 0 such that 


b 
Jim, RUF, Pans tons1) = f f, 


b 
lim R(f, Pan, Ten) =f f. 
n—oo a 


Since Weg x fr f, the sequence {R(f, Pn, Tm) }°21 does not converge. 
b) This follows from Theorem 8.23: therein, the number J satisfying (ii) was unique. 
Our condition (ii) is more stringent, so there can be at most one I satisfying it. 

c) This is almost immediate from the equivalence (ii) <= (iii) and part b): we 
leave the details to the reader. 


4.3. The Riemann Integral. 


By definition, a function f : [a,b] > R satisfying condition (ii) of Theorem 8.26 is 
Riemann integrable, and the number J associated to f is called the Riemann 
integral of f. In this language, what we have shown is that a function is Riemann 
integrable iff it is Darboux integrable, and the associated integrals are the same. 


As mentioned above, Riemann set up his integration theory using the Riemann inte- 
gral. Some contemporary texts take this approach as well. It really is a bit messier 
though: on the one hand, the business about the taggings creates another level 
of notation and another (minor, but nevertheless present) thing to worry about. 
But more significantly, the more stringent notion of convergence in the definition 
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of the Riemann integral can be hard to work with: directly showing that the com- 
position of a continuous function with a Riemann integrable function is Riemann 
integrable seems troublesome. On the other hand, there are one or two instances 
where Riemann sums are more convenient to work with than upper and lower sums. 


Example 4.2: Suppose f,g : [a,b] + R are both Darboux integrable. We wanted 
to show that f +g is also Darboux integrable...and we did, but the argument was 
slightly complicated by the fact that we had only inequalities 


L(f,P) + L(g,P) < L(f+9,P), U(f,P)+U(g,P) = Uf +9,P). 
However, for any tagging 7 of P, the Riemann sum is truly additive: 
R(f+9,P,7) = R(f,P,7) + Rg, P,7). 


Using this equality and Theorem 8.23 leads to a more graceful proof that f + g is 
integrable and i ftg= ie ft ie I encourage you to work out the details. 


Example 4.3: Let f : [a,b] > R be differentiable such that f’ is Darboux inte- 
grable. Choose a partition P = {a = 4% < 4 <... < &p_1 < Lp = D} of [a,)]. 
Apply the Mean Value Theorem to f on [2;, v;41]: there is 77 € (aj, 7:41) with 


f (xii) — f(s) = f' (27) (ti41 — 2%). 
Now {a7 er gives a tagging of P. The corresponding Riemann sum is 


R(f,P,7) = (F(a1) — F(ao)) +--+ (F(@n) — F(an-1)) = F(0) — F@). 


Thus, no matter what partition of [a,b] we choose, there is some tagging such that 


the corresponding Riemann sum for ie f’ is exactly f(b) — f(a)! Since the integral 
of an integrable function can be evaluated as the limit of any sequence of Riemann 
sums over tagged partitions of mesh approaching zero, we find that iP f' is the limit 
of a sequence each of whose terms has value exactly f(b) — f(a), and thus the limit 
is surely f(b) — f(a). This is not really so different from the proof of the supple- 
ment to the Fundamental Theorem of Calculus we gave using upper and lower sums 
(and certainly, no shorter), but I confess I find it to be a more interesting argument. 


Remark: By distinguishing between “Darboux integrable functions” and “Riemann 
integrable functions”, we are exhibiting a fastidiousness which is largely absent in 
the mathematical literature. It is more common to refer to the Riemann inte- 
gral to mean either the integral defined using either upper and lower sums and 
upper and loewr integrals or using convergence of Riemann sums as the mesh of a 
partition tends to zero. However, this ambiguity leads to things which are not com- 
pletely kosher: in the renowned text [R], W. Rudin gives the Darboux version of 
“the Riemann integral”, but then gives an exercise involving recognizing a certain 
limit as the limit of a sequence of Riemann sums and equating it with the integral 
of a certain function: he’s cheating! Let us illustrate with an example. 


Example 4.4: Compute limy_+. os Pan: 


Solution: First observe that as a consequence of Theorem 8.26, for any Darboux 
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integrable function f : [0,1] + R, we have 


lim — Heft 
noo ies 


Now observe that our limit can be ets as a special case of this: 
oe a ah n aka 1 
te a en eae el = Jin ID) 


where f(x) = sat Thus the limit is 


1 dx T 
i, > = arctan! —arctan0 = 5. 
9 z7-t+1 2 
Anyway, we have done our homework: by establishing Theorem 8.26 we have earned 
the right to use “Darboux integral” and “Riemann integral’ interchangeably. In 
fact however we will generally simply drop both names and simply speak of “in- 
tegrable functions” and “the integral”. For us, this is completely safe. However, 
as mentioned before, you should be aware that in more advanced mathematical 

analysis one studies other kinds of integrals, especially the Lebesgue integral. 


5. Lesbesgue’s Theorem 


5.1. Statement of Lebesgue’s Theorem. 


In this section we give a characterization of the Riemann-Darboux integrable func- 
tions f : [a,b] > R due to H. Lebesgue. Lebesgue’s Theorem is a powerful, definitive 
result: many of our previous results on Riemann-Darboux integrable functions are 
immediate corollaries. Here we give an unusually elementary proof of Lebesgue’s 
Theorem following lecture notes of A.R. Schep. 


For an interval J, we denote by ¢(J) its length: for all -coo <<a<b<o, 
¢((a,b)) = €((a,b)) = &((a,b]) = &((a,8]) = 6 —a, 
&((a, 00)) = &([a, 00)) = £((—00, b)) = &((—00, b]) = &((—00, 00)) = 00. 
We define a subset S C R to have measure zero if for all « > 0, there is a sequence 


{Z,} of open intervals in R such that (i) for all N > 1, yy l(I,) < €, and (ii) 
SC Ny ie In, i.e., every point of S lies in at least one of the intervals J. 


PROPOSITION 8.27. Let {5;,}°2, be a sequence of subsets of R. If each S, has 
measure zero, then so does their union S =>, Sn. 


Proor. Fixe > O0andn€Z. Since S$, has measure zero, it admits a covering 
by open intervals {Jn,.}?2, with 37), €Un,h) < g41- Then if we collect together all 
the open intervals into one big set {In,4}n,, and use the fact that sums of series 
with positive terms do not depends on the ordering of the terms, we see that 


Soin = Dna) a: sat =e. 


Further, {J,,,;} is a covering of S =U, Sp by open intervals. There is one nagging 
technical problem: the covering is not given by a sequence — i.e., not indexed by 
the positive integers — but rather by a double sequence — i.e., indexed by ordered 
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pairs (n,k) of positive integers. But this is easily solved, simply by reindexing the 
terms of a double sequence ay, via a single sequence b,, e.g. as follows: 


1,1, 41,2, 42,1, 41,3, 42,2, 43,1, 41,4, 42,3, 43,2, 44,1,41,5---- 


THEOREM 8.28. (Lebesgue) For a function f : [a,b] + R, let D(f) be the set 
of x € [a,b] at which f is discontinuous. The following are equivalent: 
(i) f is Riemann-Darbous integrable on [a, b]. 
(ii) f is bounded and D(f) has measure zero. 


Thus a function is Riemann-Darboux integrable on [a, }] iff it is bounded (which 
we already know is necessary) and its set of discontinuities is “small” in the sense 
of having measure zero. 


Exercise: Let f,g : [a,b] — R be Riemann-Darboux integrable functions. Use 
Theorem 8.28 to give a quick proof that f+g and f-g are both Riemann-Darboux 
integrable. (Suggestion: also use Proposition 8.27.) 


5.2. Preliminaries on Content Zero. 


The notion of sets of measure zero is a vitally important one in the further study 
of functions of a real variable. In fact one goes further, by assigning a measure 
(to be entirely technically precise, an outer measure) to any subset S C R as the 
infimum of all quantities )>°°_, Un) as {In}9L1 ranges over all sequences of open 
intervals with S C Qe, In. This is the beginning of a branch of real analysis 
called measure theory. In contrast, the following definition ought to be viewed 
as merely a technical tool used in the proof of Lebesgue’s Theorem. 

A subset S C R has content zero if for every € > 0, there exists N ¢ Z* and 
open intervals [1,..., Ij such that ean U;) <e and Sc Ges is 


Exercise: Show that if in the definitions of measure zero and content zero we replace 
“open interval” by “interval”, we do not change the classes of sets of measure zero 
or content zero. 


Exercise: a) Show that Q/N [0,1] has measure zero. 
b) Show that Q/N [0, 1] does not have content zero. (Suggestion: induct on N, the 
number of intervals.) 


LEMMA 8.29. Let f : [a,b] + R be non-negative and Riemann-Darbouz inte- 


grable, with ee =0. Then: 
a) For all C > 0, the set 


So = {x € [a,8] | f(x) = Ch 


has content zero. 
b) The set S = {x € [a,b] | f(x) > 0} has measure zero. 


PRooF. Fix € > 0, and choose a partition P of [a,b] such that U(f,P)) < €-C. 
Let i be the set of indices such that Sc M [x;, 2:41] is nonempty. Thus, ie I ==> 
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M; = sup(f, [ti, vi41]) > C. It follows that 
e-C>U(f,P) >>> Mi(aiz — 1) > CD) &([xi, tial). 
i€l i€l 
Thus So C Uje7[*i, %i41] and Yo ,e, C([@i, Vi41]) < €. Thus Sc has content zero. 
b) We have S =(J™, Si. Each $1 has content zero, hence certainly has measure 


zero. Now apply Proposition 8.27. 


Exercise: Let E be a subset of R, and let FE be the set of z € R such that for all 
5 > 0, there is y € E with |x — y| < 6. Show that E has content zero iff E has 
measure zero. 


5.3. Proof of Lebesgue’s Theorem. 


The proof of Lebesgue’s Theorem uses Heine-Borel (Theorem 6.17) and also the 
fact that a bounded monotone sequence is convergent (Theorem 10.11), which we 
will not discuss until Chapter 9. All in all this section should probably be omitted 
on a first reading, and only the extremely interested student need proceed. 


Step 0: Let P = {a = a < a < ... < Yn-1 < Ln = 5} be a partition of 
[a,b]. As usual, for 0 <i<n-—1, put 


m, = inf(f, [w;, ¢i41]), Mi = sup(f, [2i, vi41], 


so that 
n-1 n-1 
L(f,P) = D5 mil((i,2:41]), USP) = 5 Mie((ai, 2i41])- 
i=0 1=0 


Let vy, ® : [a,b] > R be the lower and upper step functions, i.e., y takes the value m; 
on [x;, 2441) and ® takes the value M; on [x;, 7:41). The functions y, ® are bounded 
with only finitely many discontinuities, so are Riemann-Darboux integrable by X.X; 
moreover 


[onus [e =U(f,P). 


Step 1: Suppose that f : [a,b] > R is Riemann-Darboux integrable. By X.X, 
f is bounded, so it suffices to show that the set D(f) of discontinuities of f has 
measure zero. Let {P,} be a sequence of partitions of [a,b] with P, C Px41 such 
that |P;,| > 0 (e.g. let P, be the uniform subdivision of [a,b] into 2" subintervals). 
Let yy, and ®; be the lower and upper step functions for P;,, so that for all k and 


all x € [a, b) and 
b b 
fomqcures fis 


[ques fis 


Further, since Py C Px+1 for all k, for all x € [a, 6), the sequence ¢,(z) is increasing 
and bounded above, hence convergent, say to y(a); and similarly the sequence ©; (x) 
is decreasing and bounded below, hence convergent, say to ®(a). For all x € [a,b), 
we have 


pr(a) < p(@) < f(x) < O(a) < (2), 
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b b Pb b b Ab b 
fosfesfesfrsfosfos fm 


These inequalities show that and ® are Riemann-Darboux integrable and [ i p= 


and thus 


fr = his f. Applying Lemma 8.29 to ® — y, we get that y = ® except on a set of 
measure zero. Let 


B= {xe lad] | o(0) #2) UM. 


Since F is the union of a set of measure zero with a the union of a sequence of 
finite sets, it has measure zero. We CLAIM that f is continuous at every point of 
[a, b] \ E, which will be enough to complete this direction of the proof. 

PROOF OF CLAIM Fix 2o € [a,b] \ E. Since v(x) = ®(z), there is k € Zt such 
that ©,(%0) — ye (ao) < €. Further, since ro ¢ Px, there is 6 > 0 such that ©, — y, 
is constant on the interval (2 — 6,209 + 6), and for x in this interval we have 


—€ < pr(®o) — Px (ao) < f(x) — f(wo) < Px (ao) — Px(wo) <€, 


so f is continuous at xo. 
Step 2: Suppose now that f is bounded on [a,b] and continuous on [a,b] \ E£ for a 
subset F& of measure zero. We must show that f is Riemann-Darboux integrable 
on [a,b]. Let M be such that |f(a)| < M for all x € [a,b]. Fix e€ > 0. Since E has 
measure zero, there is a sequence {J,,}°°, of open intervals covering E such that 
yr Ln) < aig. For a € [a,b] \ E, by continuity of f there iB an open interval 
J, containing @ aie that for all y,z € Jz, |f(y) — f(z)| < Wray a" Applying the 
Heine-Borel Theorem (Theorem 6.17) to the open covering {Ix} U {Je }aefas}\z Of 
[a, b], there is a finite subcovering, say {I,}f_, U {Je,}. Let P = {a = to < ti < 
. <ty =)} be the partition of [a, b] whose points are the endpoints of 11,..., IN 
and the endpoints of the intervals J;, which lie in [a,b]. or 0 < 7 < N—1, the 
subinterval (t;,¢;+1) is contained in some J, or some J,,. Let Z be the set of ¢ such 
that (¢;,ti41) is contained in J, for some k. Then 


N-1 
U(f,P) - = ST U((ti, tis) sup(f(@) — f(y) : 2,9 € [ti tis) 
i=0 
< SoM (tistisa)) -2M +3 M(tistissl spy 
re ee 
< (qqz) 2M g 5G =a" 


Since « > 0 was arbitrary, f is Riemann-Darboux integrable on [a, }]. 


6. Improper Integrals 


6.1. Basic definitions and first examples. 
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6.2. Non-negative functions. 


Things become much simpler if we restrict to functions f : [a,oo) — [0,00): in 
words, we assume that f is defined and non-negative for all sufficiently large x. 
As usual we suppose that f is integrable on [a,b] for all b > a, so we may de- 
fine F(x) = ie f for x > a. Then the improper integral ie f is convergent iff 
limz-+o0 F(x) exists. But here is the point: since f is non-negative, F’ is weakly 
increasing: indeed, for x1 < rg, F (a2) — F(a1) = Hh f- yb f= ie > 0. Now 
for any weakly increasing function F' : [a,oo) + R we have 

lim F(a) = sup(f, [a, c)). 

xr Oo 
In other words, the limit exists as a real number iff F' is bounded; otherwise, the 
limit is co: there is no oscillation! We deduce: 


PROPOSITION 8.30. Let f : [a,0o) — [0, 00) be integrable on every finite interval 
[a, N] with N >a. Then either [* f is convergent or f{~ f = ov. 


In view of Proposition 8.30, we may write the two alternatives as 
e [-* f < co (convergent) 
e [~ f =o (divergent). 


Example: Suppose we wish to compute ese ene Well, we are out of luck: this 
integral cannot be destroyed — ahem, I mean computed — by any craft that we 
here possess.* The problem is that we do not know any useful expression for the 
antiderivative of e~* (and in fact it can be shown that this antiderivative is not an 
“elementary function”). But because we are integrating a non-negative function, 
we know that the integral is either convergent or infinite. Can we at least decide 
which of these alternatives is the case? 
Yes we can. First, since we are integrating an even function, we have 


ee 2 i 2 
‘i e7 = af @e Fx, 
—oo 0 
x 


Now the function e~” = = is approaching 0 very rapidly; in fact a function like 
en = = exhibits exponential decay, and our function is even smaller than that, 


at least for sufficiently large x. So it seems like a good guess that hee e-® <0. 
Can we formalize this reasoning? 


Yes we can. First, for all x > 1, x < x”, and since (e”)’ = e” > 0, e® is in- 
creasing, so for alla > 1 e” < er, and finally, for all « > 1, er <e~*. By the 


familiar (12) property of integrals, this gives that for all N > 1, 


N ¥ N 
/ e* < / ers 
1 1 


and taking limits as N — oo we get 


lo e) % co 
i: 6r" 2& - e”. 
1 1 


4Blrond: “The Ring cannot be destroyed, Gimli, son of Gléin, by any craft that we here 
possess. The Ring was made in the fires of Mount Doom. Only there can it be unmade. It must 
be taken deep into Mordor and cast back into the fiery chasm from whence it came.” 
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This integral is much less scary, as we know an antiderivative for e~*: —e~*. Thus 


Co 2 co 1 
/ ev < / e® = —e *|" = (eM —e t= -, 
1 1 © 


Note that we replaced te e-* with ie e-®”: does that make a difference? Well, 


yes, the difference between the two quantities is precisely i, ene but this is a 
“proper integral”, hence finite, so removing it changes the value of the integral — 
which we don’t know anyway! — but not whether it converges. However we can be 
slightly more quantitative: for all « € R, x? >0 so e* <1, and thus 


1 1 
2 
i, e7” < | 1=1, 
0 0 
and putting it all together, 


O 8 Pe”. AS eee OF ke 1 
/ | ev na(fie+ | e*) <a(142) =2785.... 
—oo 0) 0 1 e 


The exact value of the integral 7s known — we just don’t possess the craft to find it: 
foe) 
/ eo? Sa Sa: &. 


This is indeed smaller than our estimate, but it is a bit disappointingly far off. 
Later in the course we will develop methods suitable for approximating f. te e-®’ to 
any desired finite degree of accuracy, and we will be able to check for instance that 
this integral agrees with \/7 to, say, 20 decimal places. 


The following simple theorem formalizes the argument we used above. 


THEOREM 8.31. (Comparison Test For Improper Integrals) Let f,g : [a,co) > 
[0, 00) be integrable on [a, N] for all N > a. Suppose f(x) < g(x) for allx >a. 
a) If fg < oo, then anges 3 <O. 
b) ae eee = 00, then Ig = 00 

Proor. By property (12) of integrals, for all N > a since f(x) < g(x) on [a, N] 
we have i f< i g. Taking limits of both sides as N — 00 gives 


(52) fret @ 


here each side is either a non-negative real number or oo. From (52) both parts 
follow: if fe g < & then Se f < oo, whereas if la f =o then i g=o~. 


THEOREM 8.32. (Limit Comparison Test For Improper Integrals) 
Let f,g : [a,co) — [0, 00) be integrable on [a, N] for all N > a. Consider condition 
(S): there exists b > a and M > 0 such that f(x) < Mg(z) for all x > b. 
a) If (S) holds and [* g < cv, then [* f < 00. 
b) If (S) holds and [* f =o, then [* g = co 
c) Suppose there exists b > a such that g(x) > 0 for alla > b and that lim, +. oo a 
L<oo. Then (S) holds. 


5In fact parts a) and b) are contrapositives of each other, hence logically equivalent. 
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d) Suppose there exists b > a such that g(x) > 0 for alla > b and that limz_,o6 oe = 
L withO< L < oo. Then 


[oe) CO 
| fee ee f g< oH. 
a a 


PRooF. For any b > a, since f and g are integrable on [a,b], we have 


fe feos fo foo, fo g<co es fo g<@~. 


a) If f(z) < Mg(x) for all x > b, then sae f< sha Mg = Mf,” g. Thus if 
Ig < 00, ig < 00, so 1a f <@ and thus finally oes f<o. 
b) Note that this is precisely the contrapositive of part a)! Or to put it in a 
slightly different way: suppose (S) holds. Seeking a contradiction, we also suppose 
4 hee g < oo. Then by part a), EE f < ov, contradiction. 

c) Since limz-.0 fa = = L < ow, there is b > a such that for all x > b, fa <D+1. 
Thus for all x > b we have f(x) < (L+ 1)g(z), so (S) holds. 


d) Suppose limz +. ia = = L € (0,0). By part c), (S) holds, so by part a), if 


ie g < ©, then ve f <. Moreover, since L 4 0, limz-+, eal = + € (0,00). So 


part c) applies with the roles of f and g reversed: if i f <c then 1 ae g<o. 


Although from a strictly logical perspective part d) of Theorem 8.32 is the weakest, 
it is the most useful in practice. 


7. Some Complements 


Riemann was the first to give a complete, careful treatment of integration as a pro- 
cess that applies to a class of functions containing continuous functions and yields 
the Fundamental Theorem of Calculus: he was certainly not the last. 


In more advanced analysis courses one studies the Lebesgue integral. This is 
a generalization of the Riemann integral that, extremely roughly, is modelled on 
the idea of approximations in small ranges of y-values rather than small ranges of 
x-values. Suppose for instance that f : [0,1] > R has a finite range yi,..., yn. 


Then the idea is that ie f is determined by the size of each of the sets f~+(y;) on 
which f takes the value y;. For instance this is the perspective of the expected value 
in probability theory: given a function defined on a probability space, its expected 
value is equal to the sum of each of the values at each possible outcome multipled 
by the probability that that outcome occurs. Notice that when f is a step function, 
the sets f~'(y:) are just intervals, and the “size” of an interval (a,b), [a,b), (a, b] or 
[a, 6] is just its length b — a. However, a general function f : [0,1] — R with finite 
image need not have the sets f—'(y;) be intervals, so the first and hardest step in 
Lebesgue’s theory is a rigorous definition of the measure of a much more general 
class of subsets of [a,b]. This has grown into a branch of mathematics in its own 
right, measure theory, which is probably at least as important as the integra- 
tion theory it yields. However, Lebesgue’s integral has many technical advantages 
over the Riemann integral: the supply of Lebesgue integrable functions is strictly 
larger, and in particular is much more stable under limiting operations. Later we 
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will study sequences of functions f,, : [a,b] + R and see that a crucial issue is the 
permissibility of the interchange of limit and integral: i.c., is 


b n 
lim fc i lim fr? 
n—-oo a a noo 

For the Riemann integral it is difficult to prove general theorems asserting that the 
left hand side exists, a phenomenon which evidently makes the identity hard to 
prove! Probably the real showpiece of Lebesgue’s theory is that useful versions of 
such convergence theorems become available without too much pain. 


In classical probability theory one encounters both “discrete” and “continuous” 
probability distributions. The idea of a continuous probability distribution on [a, }] 
can be modelled by a non-negative integrable function f : [a,b] > R such that 


hs f =1. This is studied briefly in the next chapter. Lebesgue’s theory allows for 
distributions like the Dirac delta function, a unit mass concentrated at a single 
point. It turns out that one can model such discrete distributions using a simpler 
theory than Lebesgue’s theory due to the Dutch mathematician T.J. Stieltjes (born 
in 1856; Riemann was born in 1826, Lebesgue in 1875). This is a relatively mild 
souping up of Riemann’s theory which views the dx as referring to the function 
f(a) = x and replaces it by dg for a more general function g : [a,b] > R. Some 
undergraduate texts develop this “Riemann-Stieltjes integral”, most notably [R]. 
But this integral adds one extra layer of technical complication to what is already 
the most technical part of the course, so I feel like in a first course like this one 
should stick to the Riemann integral. 


In the twentieth century various generalizations of the Riemann integral were devel- 
oped that are much more closely related to the Riemann integral than the Lebesgue 
integral — avoiding any need to develop measure theory — but are yet more power- 
ful than the Lebesgue integral: i.e., with a larger class of integral functions, good 
convergence theorems, and a simple general form of the Fundamental Theorem of 
calculus. This integral is now called the Kurzweil-Henstock integral although 
it was also developed by Denjoy, Perron and others. This theory however is not so 
widely known, whereas every serious student of mathematics studies the Lebesgue 
integral. As alluded to above, the reason is probably that the measure theory, while 
a lot to swallow the first time around, is in fact a highly important topic (in par- 
ticular making a profound connection between analysis and geometry) in its own 
right. It is also extremely general, making sense in contexts far beyond intervals 
on the real line, whereas the Kurzweil-Henstock integral is more limited in scope. 


Aside from developing his new and more sophisticated theory of integration, Lebesgue 
did much insightful work in the more classical context, as for instance his remark- 
able characterization of Riemann integrable functions presented above. I want to 
mention a further achievement of his which has been almost forgotten until re- 
cently: from the perspective of differential calculus, a natural — but very difficult — 
question is Which functions are derivatives? We now know that in particular 
every continuous function f : [a,b] + R is the derivative of some other function 
F’, but in order to construct the antiderivative we had to develop the theory of 
the Riemann integral. Is it possible to construct an antiderivative of an arbitrary 
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continuous function f in some other way than as A f? (I don’t mean is there some 


other function which is antiderivative: as we know, iP f +C is the most general 
antiderivative of f. But for instance we certainly don’t need an integration theory 
to know that polynomials have antiderivatives: we can just write them down. The 
question is whether one can find an antiderivative without expressing it in terms of 
an integration process.) 


Lebesgue gave a considerably more elementary construction of antiderivatives of 
every continuous function. The basic idea is shockingly simple: on the one hand it 
is easy to write down an antiderivative of a piecewise linear function: it is simply 
an appropriate piecewise quadratic function. On the other hand every continu- 
ous function can be well approximated by piecewise linear functions, so that one 
can write down the antiderivative of any continuous function as a suitable limit of 
piecewise quadratic functions. A nice modern exposition is given in [Be13]. 


CHAPTER 9 


Integral Miscellany 


1. The Mean Value Therem for Integrals 


THEOREM 9.1. Let f,g : [a,b] ~ R. Suppose that f is continuous and g is 
integrable and non-negative. Then there is c € [a,b] such that 


(53) ih “= 10 i 4 


PRooF. By the Extreme Value Theorem, f assumes a minimum value m and 
a maximum value M on [a,}]. Put I = nk g; since g is non-negative, so is J. We 
have 


(54) m= [mgs [fox [ Mg= ut, 


Case 1: Suppose J = 0. Then by (54), for all c € [a,b] we have 


Vi fg =9= fe) ro fo 
Case 2: Suppose J # 0. Then dividing (54) through by I gives 
b 
a tI 
I 


m< <M. 


Thus Sa ee, lies between the minimum and maximum values of the continuous func- 
tion f : [a, a + R, so by the Intermediate Value Theorem there is c € [a,b] such 


‘ 
that f(c) = oo . Multiplying through by ie g, we get the desired result. 


Exercise 9.1.1: Show that in the setting of Theorem 9.1, we may take c € (a,b). 


Exercise 9.1.2: Show by example that the conclusion of Theorem 9.1 becomes false 
— even when g = 1 — if the hypothesis on continuity of f is dropped. 


Exercise 9.1.3: Suppose that f : [a,b] — R is differentiable and f’ is continu- 
ous on [a,b]. Deduce the Mean Value Theorem for f from the Mean Value Theoem 
for Integrals and the Fundamental Theorem of Calculus.+ 


2. Some Antidifferentiation Techniques 


2.1. Change of Variables. 


lSince the full Mean Value Theorem does not require continuity of f’, this is not so exciting. 
But we want to keep track of the logical relations among these important theorems. 
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2.2. Integration By Parts. 


Recall the product rule: if f,g: J — R are differentiable, then so is fg and 
(f9)' = fot fo’. 

Equivalently, 
fo! = (fg)' — f'9. 

We have already (!) essentially derived Integration by Parts. 


THEOREM 9.2. (Integration by Parts) Let f,g : I > R be continuously differ- 
entiable functions: 1.e., f’ and g’ are defined and continuous on I. 
a) We have f fg’ = fg —f f’g, in the sense that subtracting from fg any anti- 
derivative of f'g gives an antiderivative of fq’. 
b) If [a,b] C I then 


b b 
/ fa! = f()a(b) — Fla)g(a) - / f'g. 


Exercise 9.2.1: a) Prove Theorem 9.2. (Yes, the point is that it’s easy.) 
b) Can the hypothesis on continuous differentiability of f and g be weakened? 


Exercise 9.2.2: Use Theorem 9.2a) to find antiderivatives for the following func- 
tions. 

a) ze” forl1<n<6. 

b) loga. 

c) arctan x. 

d) xcosz. 

e) e” sina. 

f) sin® x. 

g) sec? x, given that log(sec x + tan x) is antiderivative of sec x. 


Exercise 9.2.3: a) Show that for each n € Z*, there is a unique monic (= leading 
coefficient 1) polynomial P,(x) such that 4 (P,(x)e”) = a"e". 

b) Can you observe/prove anything about the other coefficients of P,,(a)? (If you 
did part a) of the preceding exercise, you should be able to find patterns in at least 
three of the non-leading coefficients. ) 


Exercise 9.2.4: Use Theorem 9.2a) to derive the following reduction formulas: 
here m,n € Z* and a,b,c ER. 


a) 
1 os F n—-1 * 
[cost = —cos”~!asina + cos”—? 
n n 


b) 


i 1 _ x 2n—3 i 1 
(x? + a7)” — 2a2(n — 1)(a? + a?)r-1 2a?(n — 1) (x2 + a2)r-1" 
c) 

—1 


sin” az cos” ax = ———— sin”! ax cos"*! ax + 
a(m +n) m+n 


m—1 


i: sin”? ax cos” ax 
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a , 
= —— sin”! ax cos ax + 


3 ies. eet pow ax cos”? ax. 
a(m +n) m+n 
In this text we are not much interested in antidifferentiation techniques per se — we 
regard this as being somewhat forward-looking, since computer algebra packages 
are now better at this than any human ever could be — so most of our interest in 
integration by parts comes from part b) of Theorem 9.2. In fact this result, simple 
though it is, is a remarkably powerful tool throughout honors calculus and analysis. 


PROPOSITION 9.3. For alln EN, jes ze "dx = nl. 


Proor. By induction on n. Base case (n = 0): 
So e7® = —e7*|8° = —e- — (-e°) = -0 - (-1) =1=0! 
Induction step: let n € N and assume ee z”e~*dx = n!. Now to make progress in 


evaluating ie x"tle-*dz, we integrate by parts, taking u = 2" +1, dv = e7*dz. 
Then du = (n+ 1)a"dz, v =e7*, and 


i. a™tle*de = (n+ 1)x"e*|0° — | (—e-*(n + 1)a”)dx 
0 0 


=(0-0)+ (n+ » f ae "dx = (n+1)n! =(n+1)! 
0 
2.3. Integration of Rational Functions. 


3. Approximate Integration 


Despite the emphasis on integration (more precisely, antidifferentiation!) techniques 
in a typical freshman calculus class, it is a dirty secret of the trade that in practice 
many functions you wish to integrate do not have an “elementary” antiderivative, 
ie., one that can be written in (finitely many) terms of the elementary functions one 
learns about in precalculus mathematics. Thus one wants methods for evaluating 
definite integrals other than the Fundamental Theorem of Calculus. In practice it 
would often be sufficient to approximate i f rather than know it exactly.” 


THEOREM 9.4. (Endpoint Approximation Theorem) Let f : [a,b] > R be dif- 
ferentiable with bounded derivative: there is M > 0 such that |f'(a)| < M for 
all x € [a,b]. Forn € Zt, let Ly(f) be the left endpoint Riemann sum ob- 
tained by dividing [a,b] into n equally spaced subintervals: thus for0 <i<n-1, 
aj* =at+i(=*) and L,(f) = ons f(ai*)=*. Then 


[ 4- tata] < (RP) 2 


PRooF. Step 1: We establish the result for n = 1. Note that Li(f) = (b- 
a) f(a). By the Racetrack Principle, for all x € [a,b] we have 


—M(a — a) + f(a) < f(x) < M(@w—a) + fla) 


[c Vex fe ms firs for (e —a) + f(a)). 


2It would be reasonable to argue that if one can approximate the real number iis f to any 


and thus 


degree of accuracy, then in some sense one does know it exactly. 
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Step 2: Let n € Z*. Then 


b eet Tit b-a eae 
ft t0 S(f- f- fa" \(— Nish 
@ i=0 \" i 


Step 1 applies to each term in the latter sum to give 


b 


Exercise 9.3.1: Show that Theorem 9.4 holds verbatim for with left endpoint sums 
replaced by right endpoint sums R,,(f). 


Exercise 9.3.2: a) Suppose f : [a,b] + R is increasing. Show that 


b 
0< i: f-Inlf) < (FQ) - f(@)(0-a)=. 


n 


b) Derive a similar result to part a) for the right endpoint sum R,,(f). 
c) Derive similar results to parts a) and b) if f is decreasing. 


In view of the preceding exercise there is certainly no reason to prefer left end- 
point sums over right endpoint sums. In fact, if you stare at pictures of left and 
right endpoint sums for a while, eventually it will occur to you to take the average 
of the two of them: with A = boa we get 
(55) 

1 


1 
Tn(f) = En(f)+ Ralf) = 5 fa) + f(a+A)+ f(at2A)..+ flat (n-1)A)+ 500) 
On each subinterval [x;,7;41] the average of the left and right endpoint sums is 
a = oe (t:41 —2;). Notice that this is the 


area of the trapezoid whose upper edge is the line segment which joins (2;, f(x;)) 


to (w41, f(i41). For this reason the approximation T;,(f) to di f is often called 
the trapezoidal rule. That’s cute, but a more useful way of thinking about T;,(/) 
is that it is obtained by linearly interpolating f on each subinterval [;, 7:41] 
via the line which matches f at the two endpoints of the interval. 


Thinking of the trapezoidal rule this way suggests that T,,(f) should, at least for 
nice f and large n, a better approximation to ihe f than either the left or right 
endpoint sums. For instance, if f is linear then on each subinterval we are approx- 
imating f by itself and thus T,,(f) = iL f. This was certainly not the case for 
the endpoint rules: in fact, our proof of Theorem 9.4 showed that for each fixed 
M, taking f to be linear with slope M (or —M) was the worst case scenario: if 
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we approximate a linear function ¢(x) with slope +M on the interval [a,b] by the 
horizontal line given by the left endpoint (say), then we may as well assume that 


€(a) = 0 and then we are approximating ¢ by the zero function, whereas fre is 
the area of the triangle with base (b— a) and height M(b— a), hence “(b — a)?. 
Another motivation is that if f is increasing, then L,,(f) is a lower estimate and 
R,(f) is an upper estimate, so averaging the two of them gives something which is 


closer to the true value. 


The following result confirms our intuitions. 


THEOREM 9.5. (Trapezoidal Approximation Theorem) Let f : [a,b] > R be 
twice differentiable with bounded second derivative: there is M > 0 such that 


|f"(x)| <M for all x € [a,b]. Then for alln € Zt we have 


[s-mu < (oo) a 


Proor. Put A = 2%, and for i =0,...,n—1, let 4 =a+iA. Also put 


n? 


t 


a AIR, w= F@+seto)- fF 


As motivation for this definition we observe that 
n-1 b 
(56) (A) =Tal)- fF. 
i=0 2 


We have y;(0) = 0 and, using the Fundamental Theorem of Calculus, 


Metis Bo 1 .)4 . 4 N t La re ral — 1 . 4 \ t 
Yi 9 (f (7) T f(% v t)) I af (aj+t) f(aitt) 5 (F(a) f(xi+t)) ! 9 
so yi (0) = 0 and 

a hy | me jl a eee 1 
ay 5) f(ait+t4 af (ai + t) 4 atl (aj +t) tf" (xi +t) 
Put 


A= inf(f",[a,b]) B=sup(f”, [a, }]). 
Then for all 0 <i <n—1 and t € (0, A] we have 


A B 
—t < y(t) < —t. 
5 <i (t) < 5 


Since y/(0) = 0, integrating and applying the Racetrack Principle gives 
Ais / B iy 
—t' <y,(t)< —t; 
qt svt) s eS 
doing it again (since y;(0) = 0) and plugging in t = A gives 
A B 
— A? < y;(A) < =A? 
po SA) Sp 
Summing these inequalities from i = 0 to n — 1 and using (56) we get 


ee : B 3 
—— cies | = <— : 
24 n<Th(f) [ f< 4 n 


190 9. INTEGRAL MISCELLANY 


Substituting in A = 2—* gives 


(A028) cr first) 


Since |A|,|B| < M, we get the desired result: 
(b-a)®M\ 1 
Tn( awe 
[ f- p| s (SoM 12 ne 


Exercise 9.3.3: In the setting of Theorem 9.5, suppose f” is continuous on [a, d}. 
a) Show: there is c € [a,b] such that T,(f) — for = (4F°) : 


n° 
b) Show: if f is linear, the trapezoidal rule is exact: ie f =T,(f) for alln € Z*. 
c) Show: if f = 2? + bx +c, the trapezoidal rule is never exact: ies fAT,AS). 


Exercise 9.3.4: Show: if f : [a,b] + R is convex, then T,,(f) > fe fi 


Looking back at the formula (55) for the trapezoidal rule we notice that we have 
given equal weights to all the interior sample points but only half as much weight 
to the two endpoints. This is an instance of a heuristic in statistical reasoning: 
extremal sample points are not as reliable as interior points. This suggests a differ- 
ent kind of approximation scheme: rather than averaging endpoint Riemann sums, 
let’s consider the Riemann sums in which each sample point x7 is the midpoint 
of the subinterval [x;,x;4,1]. Dividing the interval [a, b] into n subintervals of equal 
width A = fon as usual, this leads us to the midpoint rule 


Exercise 9.3.5: a) Show: if f is linear, the midpoint rule is exact: if? f =M,(f). 
b) Show: if f : [a,b] + R is convex, then M,(f) < if he 
The following result concerning the midpoint rule is somewhat surprising: it gives 
a sense in which the midpoint rule is twice as good as the trapezoidal rule. 


THEOREM 9.6. (Midpoint Approximation Theorem) Let f : [a,b] > R be twice 
differentiable with bounded second derivative: there is M > 0 such that | f"(x)| <M 
for all x € [a,b]. Then for alln € Z* we have 


[r-anco s (a) 


PROOF. Put AS 4y tors 0)... ,n—1, let aj=at+(its 5)A. Also put 


xitt 


W; : (0, A/2] +R, w(t) =f f —2tf (ai). 


at 
As motivation for this definition we observe that 


n-1 b 
(57) SHA) = f F- Mal) 
i=0 oe 
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We have W;(0) = 0, and using the Fundamental Theorem of Calculus, 
Wi (t) = fw +t) — fw — t)(-1) - 2f (ai) = (f(a + t) + f(a — t)) — 2f (ai). 


W"(t) = filer +t) — f'(ai—?). 
Applying the Mean Value Theorem to f’ on [x; — t, 2; + t] we get a point x,y € 
(a; —t,a; +t) such that 
(a, +t) - (a, —t wit 
Hn) = Lt = Me) WO 
2t 2t 


or 

Wi (t) = 2tf" (wit). 
Put 

A= inf(f",[a,b]) B=sup(f”, [a, }]). 

Then for al0 <i<n—Jlandte (0, 4] we have 

2tA < W(t) < 2B. 
Integrate and apply the Racetrack Principle twice as in the proof of Theorem 9.5, 
and plug in t = $ to get 

A B 

(58) a’ < ;(A/2) < ogo 


Summing (58) from i = 0 to n — 1, using (57) and substituting A = —* gives 


(eas 1 ee ie pes 7 


Since |A|,|B] < M, we get the desired result: 
[ism 


For integrable f : [a,b] > R and n € Z*, we define Simpson’s Rule 


: 2 1 
San(f) = gMn(f) + gInl(f). 
Thus Son(f) is a weighted average of the midpoint rule and the trapezoidal rule. 
Since our previous results suggest that the midpoint rule is “twice as good” as the 


trapezoidal rule, it makes some vague sense to weight in this way. 


Exercise: a) Show that for any a,b € R such that a+b = 1, aM, (f)+0T,(f) > he f. 
b) Deduce that Son(f) > f? f. 


Let us better justify Simpson’s Rule. As with the other approximation rules, it 
is really a composite rule, i.e., it is based on iterating one simple approximation on 
many equally spaced subintervals. So let’s first concentrate on S'9(f). We have 


S2(f) = Mol f) + STH) = F((6— a) (22) + FOE 6 _ ay) 


= "=" (Fe) +4) 410), 
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Since we are dividing [a, b] into two subintervals, the width of each is A = bea so 
with v9 =a, 71 =a+A, 72 =a+2A =), we have 


(1) = (Fao) + AF a) + F22)) 


For n € Z*, putting A = baa Lo =a, 41 =at+A,...,%n-1a+ (2n-1)A, rz, = b, 
we get 


Son(f) = > (F(20) + 4f(ai) + flee) + f(w2) + 4f(@s) + F(@a) + F(aa) +... + £0) 
= F (Feo) + 4fler) + 2f lea) + +4 (nt) + Sle. 


LEMMA 9.7. For any quadratic function f(x) = Ar? +Bx+C, Simpson’s Rule 
is exact: fie = Son(f) for alln € Zt. 

Proor. Let A = one By splitting up [a,b] into n pairs of subintervals, it suf- 
fices to show he f = S2(f). This is done by a direct, if unenlightening, calculation: 


whereas 


Thus whereas the endpoint rule is an approximation by constant functions and 
the trapezoidal rule is an approximation by linear functions, Simpson’s rule is an 
approximation by quadratic functions. We may therefore expect it to be more accu- 
rate than either the Trapezoidal or Midpoint Rules. The following exercise (which 
again, can be solved by a direct, if unenlightening, calculation), shows that it is 
even a little better than we might have expected. 


Exercise 9.3.6: Suppose f is a cubic polynomial. Show that Simpson’s Rule is 
exact: Son(f) = ay f. 


Since Simpson’s Rule is exact for polynomials of degree at most 3 and a func- 
tion f is a polynomial of degree at most three if and only if f“ = 0, it stands to 
reason that Simpson’s Rule will be a better or worse approximation according to 
the magnitude of f on [a,b]. The following result confirms this. 
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THEOREM 9.8. (Simpson Approximation Theorem) Let f : [a,b] + R be four 
times differentiable with bounded fourth derivative: there is M > 0 such that 
|f(a)| < M for all x € [a,b]. Then for all even n € Z+ we have 


[ir su) «(ONS 


Proor. For 0 <i< §—1, let a; =a+ (21+ 1)A and put 


1 att 
BOA SR, B= GGA) Ee ED) / f 


3 it 
As motivation for this definition we observe that 
n-1 b 
(59) Yat) = suf fF 
i=0 
We have ©;(0) = 0 and using the Fundamental Theorem of Calculus, 
Dit) = StF (e+ F (wet) +5 (Fei +4f (ei) +f att) —FleiHt)- fai) 
= FU (ei +8) fe — 0) — FFs - 8) — 2 le) + Fler +9) 
So /(0) = 0 and 
W 1 " WW 1 / 7 2 / 2 / 
P(t) = BAP (ei + Fai tt) + oF (ett) — fi (ai-t))+ sf (zi-t)— 3 fF (zi tt) 
= sth tt) + f" (a; —t)) (F(a: +t) — f(a; —t)) 
So (0) = 0 and 
mw 1 my mw 1 W wW 1 " 1 " 
O;'(t) = gilf (xi+t)—f"(ai-t))4 3 (f (xit+t)+f"(xi-t)) af (xi+t) af (x;—-t) 


= Sr" +8) = Fi), 


So 6/’(0) = 0. Applying the Mean Value Theorem to f’” on [x; — t,x; + t], there 
is € € (a; —t,a; +t) such that 


wi +t)— f"(ai—t) 


3 fe 
~ 6! (t)/?? = = f(n). 
Sr (0/ : f(n) 
Let A = inf(f™, [a, b]), B = sup(f™, [a, b]), so for all i and all ¢ € [0, A] we have 
2A 2B» 


Sa Ot) 
3 3 


Integrate and apply the Racetrack Principle three times and plug in t = A to get 


A B 
60 — A? < (A) < —A*. 
(60) 994 = 8i(4) S op 
Summing (60) from i = 0 to $ — 1,° using (59) and substituting A = ba gives 


3Note that there are 3 terms here — half as many as in the previous results. This gives rise 


to an extra factor of 4. 
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Exercise 9.3.7: a) In the setting of the Trapezoidal Rule, suppose moreover that 
f” is continuous on [a,b]. Adapt the proof of Theorem 9.5 to show that there is 


n € [a,b] such that 
a ory 7 (Geatsen) - 


b) Derive similar “error equalities” for the Endpoint, Midpoint and Simpson rules. 


The proofs of Theorems 9.5, 9.6 and 9.8 are taken from [BS, Appendix D]. They 
are admirably down-to-earth, using only the Mean Value Theorem and the Funda- 
mental Theorem of Calculus. However it must be admitted that they are rather 
mysterious. 


Our motivations for considering the various “rules” were also a little light, but 
I hope the reader can see that they have something to do with polynomial in- 
terpolation. This will be taken up in more detail in Chapter 12. For now we 
just mention the sobering fact that all these rules — and higher-degree analogues 
of them — were already known to Newton, developed in jointly with his younger 
collaborator Roger Cotes in the early years of the 18th century. 


4. Integral Inequalities 


THEOREM 9.9. (The Hermite-Hadamard Inequality) Let f : [a,b] + R be convex 
and continuous. Then 


a+b\_ Sif — fa +f) 
r( 2 \spts 2 : 


Proor. Let s(x) = f(2¢2)+m(x— 4°) be a supporting line for f at « = “4°, 
and let S(x) = f(a)4 (42) (x — a) be the secant line, so 
(61) Va € [a,b], s(x) < f(x) < S(z). 
Integrating these inequalities and dividing by b — a we get 


Now we have 
I s(a) [Ee + m(a — +4) a+b m : a+b 

= = } £ 
b-a b-a 


a+b m bP a atb 
b - 0= 
7) “(5 i ae “)) Meee 


fs(ede  Se(f@) + (L4H) (@-a))de pa) + 
b-a = b-a = 2 

Ses Les 

b-a b-a 

atb) Sef < f@+S0) 
2 b-a 2 


=F 


Substituting these evaluations of and into (61) gives 


f( 
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Exercise: Show that the hypothesis of continuity in Theorem 9.9 is not necessary: 
the inequality holds for any convex f : [a,b] > R. 


Let [a, b] be a closed interval, and let P : [a,b] + [0, 00) be a probability density: 
i.e., P is integrable on [a,b] and [PP = 1. For an integrable function f : [a,b] > R, 
we define the expected value 


b 
Bp) = | f(a) P(x)dx 


THEOREM 9.10. (Jensen’s Integral Inequality) Let P : [a,b] — [0,00) be a 
probability density function, y : [c,d] + R be convex, and let f : [a,b] — [c,d] be 
integrable. Then E(f) € [c,d] and 


P(E(f)) < E(v(f))- 


Proor. Since f : [a,b] > [c,d], we have 


ox [orl ode sf f(a)? ns fara) =a 


so ib f(x)P(x)dz = E(f) € [c,d] and thus y(E(f)) is defined. Now put xo = E(f) 
and let s(#) = ma + B be a supporting line for the convex function ~ at xo, so 
s(x) < y(a) for all x € [c,d] and s(xo) = y(ao). Now 


b b 
E(y(f)) = te olf (0)) Pade > / (mf (co) + B)P(a)dx 


=m fi f(a) P(a)dz + B= mE(f)+ B= ma + b= v(X0) = o(E(f)). 


THEOREM 9.11 (Hélder’s Integral Inequality). Let p,q € R° be such that 


++4=1. Let f,g: [a,b] + R be Riemann integrable functions. Then 


[ss (f'ur)’ (fer) 


PROOF. Step 1: Suppose te |f|? =0. Let My and M, be upper bound for f 
and g on [a,b]. By Lemma 8.29, the set of x € [a,b] such that |f|? > € has content 
zero: thus, for every 6 > 0 there is a finite collection of subintervals of [a, b], of total 
length at most 6, such that on the complement of those subintervals | f|? <«. On 


(62) 


this complement, |fg| < > My and thus the sum of the integrals of |fg| over the 


complement is at most (b — ae? My. The sum of the Riemann integrals over the 
subintervals of total length 6 of |fg| is at most 6M; Mg, so altogether we get 


b 
if fois f Ifgl < (b—a)e® M, + 6MyM,. 


Since all the other quantities are fixed and ¢, 6 are arbitrary, we deduce | fe fg| =0. 
Thus both sides of (62) are zero in this case and the inequality holds. A similar 
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argument works if ie \g|? = 0. Henceforth we may suppose 


b b 
es) FIP > 0, i=} Ig\? > 0. 


7 1 1 
Step 2: Put f = F/T; and g = g/Ij. Then by Young’s Inequality, 


if jas [vials f° (2, at) teow 


ba i i 
Multiplying through by [7 Ig = (r IrlP) 7 Cr isl?) “ gives the desired result. 


Remark: Admittedly Step 1 of the above proof is rather technical. To an extent 
this points to the finickiness of the class of Riemann integrable functions. The stu- 
dent who continues on will learn the corresponding result for the Lebesgue integral, 
in which the proof of this step is immediate. Alternately, the student may wish 
to restrict to continuous f and g, since a non-negative continuous function f with 
i f =0 must be identically zero. 


The case p = q = 2 of Theorem 9.11 is important enough to be restated as a 
result in its own right. 


COROLLARY 9.12 (Cauchy-Schwarz Integral Inequality). Let f,g : [a,b] > R 
be Riemann integrable functions. Then 


(f'n) < fr fe 


5. The Riemann-Lebesgue Lemma 


THEOREM 9.13. (Riemann-Lebesgue Lemma) Let f : [a,b] > R be Riemann 
integrable. Then: 


a) lim)-+00 Ie f(x) cos(Ax)dx = 0. 
b) lim y-+00 hs f(x) sin(Ax)dx = 0. 


Proor. As the reader surely expects, the arguments for parts a) and b) are 
virtually identical, so we will establish part a) and leave part b) as an exercise. 

Here is the dea of the proof: if f(x) = C isa constant function, then ie C'cos(Az) 
oscillates increasingly rapidly as X increases. If the length of the interval a — b were 
a precise multiple of the period = then we get several “complete sine waves” and 
the positive and negative area cancel out exactly, making the integral exactly zero. 
As increases the remaining “incomplete sine wave” is a bounded function living 
on a smaller and smaller subinterval, so its contribution to the integral goes to zero. 

A similar argument holds for step functions, which become constant functions 
when split up into appropriate subintervals. And now the key: the integrable func- 
tions are precisely those which can be well approximated by step functions in the 
sense that the integral of the difference can be made arbitrarily small. 

Okay, let’s see the details: fix € > 0. By Darboux’s Criterion, there is a partition 
P ={a=% <2 <...< Ly_1 < Lp = 5} of [a,b] such that 


b 
0< 14) < | f<5. 
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Let g be the step function which is constantly equal to m; = inf(f, [v;,2i41]) on 
the subinterval [x;, 7:41) of [a,b], so g < f and Lg = L(f,P), so 


o< fu- NS 


Nim 


b 
) cos(Ax) dx ah |f (a x)|| cos(Ax)|da + i: g(x) cos(Ax)dx 
2 n-1 Lit. Z n-1 m 
(63) < 5 + ye m, cos(Ar)da} < 5 + » 5 (sin(Avi41) — sin(Aa;))]. 
i=0 7H i=0 


Here we have a lot of expressions of the form |sin(A) —sin(B)| for which an obvious 
upper bound is 2. Using this, the last oer in (63) is at most 

6. 2, t Ima 

a 
But this inequality holds for any \ > 0, so taking X sufficiently large we can make 


the last term at most 5 and thus lie f(a) cos(Ax)daz| < €. 


CHAPTER 10 


Infinite Sequences 


Let X be a set. An infinite sequence in X is given by a function 2, : Z+ > X. 
Less formally but entirely equivalently, we are getting an ordered infinite list of 
elements of X: 21, %2,2%3,...,%n,-.-. Note that the function is not required to be 
injective: i.e., we may have x; = 2; for i ~ j. In fact, a simple but important ex- 
ample of a sequence is a constant sequence, in which we fix some element x € X 
and take x, = x for all n. 


The notion of an infinite sequence in a general set X really is natural and important 
throughout mathematics: for instance, if X = {0,1} then we are considering infi- 
nite sequences of binary numbers, a concept which comes up naturally in computer 
science and probability theory, among other places. But here we will focus on real 
infinite sequences Zpyjier : Z‘ 4 R. In place of x, we will write {x,}°C, or 
even, by a slight abuse of notation, x. 


We say that an infinite sequence a, converges to a real number L if for all e > 0 
there exists a positive integer N such that for all n > N, we have |a,, — L| < e. 
A sequence is said to be convergent if it converges to some L € R and other- 
wise divergent. Further, we say a sequence a,, diverges to infinity — and write 
limn+soo Gn = CO Or dn — © — if for all M € R there exists N € Z* such that 
n>N = > a,>M. Finally, we define divergence to negative infinity: I leave it 
to you to write out the definition. 


This concept is strongly reminiscent of that of the limit of a function f : [1,co) > R 
as x approaches infinity. In fact, it is more than reminiscent: there is a direct con- 
nection. If limz-,.. f(z) = L, then if we form the sequence x, = f(n), then it 
follows that limp. %, = L. If x, = f(x) for a function f which is continuous — or 
better, differentiable — then the methods of calculus can often be brought to bear 
to analyze the limiting behavior of zy. 

Given a sequence {x,,}, we say that a function f : [1,oo) > R interpolates f 
if f(n) =a, for alln € Z™. 


Example: Supose x, = log ”. Then f(x) = == interpolates the sequence, and 


It follows that x, — 0. 


Exercise: Let {an}92, be a real sequence. Define f : [1,00) — R as follows: 
for c € [n,n+1), f(z) =(n+1—2Z)a, + (@— nN)an41. 
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a) Take, for example, a, =n”, and sketch the graph of f. 
b) Show that the sequence lim,_,.. f(x) exists iff the sequence {a,,} is convergent, 
and if so the limits are the same. 


The previous exercise shows that in principle every infinite sequence {a,,} can be 
interpolated by a continuous function. The given f is piecewise linear but gener- 
ally not differentiable at integer values. However, with only a little more trouble we 
could “round off the corners” and find a differentiable function f which interpolates 
{an}. But in practice this is only useful if the interpolating function f is simple 
enough to have a known limiting behavior at infinity. Many sequences which come 
up in practice cannot be interpolated in a useful way. 


Example (Fibonacci Sequence) 
Example (Newton’s method sequences). 


In fact we will be most interested in sequences of finite sums. For instance, let 


el 1 1 
an = =1+=4 +, 
: v 2 n 
ab 
and let 
ST 1 1 
b= galt ar) 


What is the limiting behavior of a, and b,? In fact it turns out that a, — oo and 
2 

by, — “7: whatever is happening here is rather clearly beyond the tools we have at 

the moment! So we will need to develop new tools. 


1. Summation by Parts 


LEMMA 10.1. (Summation by Parts) Let {a,,} and {b,,} be two sequences. Then 
for allm <n we have 


ye ap (be+1 — be) = (Qn41bn41 — Gmbm) + S> (Qr41 — Ak) be-41- 
k=m k=m 
PROOF. 


n 


S 5 an (bes — Ok) = dmbmtr +++. + Onda — (dmbm +++. + nbn) 


k=m 


= Onbn+1 Amdm ((Qm41 Qm)bm+1 wet koh (Gn = An—1)bn) 
n—-1 
= Andn+1 = Amn a= pt (Qk44 = Ap )on44 
k=m 
= Andn+1 Amn t (Qn41 Gn)On+1 = y= (Qp41 = ap) bp41 


k=m 


n 
= An41bn41 — Gm -5 (Qk41 — Ok) De+1- 


k=m 
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The proof of Lemma 10.1 could hardly be more elementary or straightforward: lit- 
erally all we are doing is regrouping some finite sums. Nevertheless the human 
mind strives for understanding and rebels against its absence: without any further 
explanation, summation by parts would seem (to me at least) very mysterious. 


The point is that Lemma 10.1 is a discrete analogue of integration by parts: 


b b 
/ fa! = f()9(b) — fla)g(a) - | fg. 


Just as integration by parts is a fundamental tool in the study of integrals — and, 
after taking limits, improper integrals — summation by parts is a fundamental tool 
in the study of finite sums — and, after taking limits, infinite series. 

Whereas for integration by parts there really is just the one identity, recalled 
above, for summation by parts, there are several alternate formulas and special 
cases, one of which is given in the following exercise. So really one should remem- 
ber the idea — starting with a sum of the form 7, dnbn, there are several identities 
involving expressions in which we have discretely differentiated {a,,} and discretely 
integrated {b,,}, or vice versa — rather than any one specific formula. 


Exercise: Let {an}°2, and {b,}9°, be sequences, and for N € Zt, put A, = 
re an, Bn = Ty bn, Ao = Bo = 0. Show that 


N N-1 
(64) So anbn = > An(bn — bnti) + Anon. 
n=1 n=1 


PROPOSITION 10.2. (Abel’s Lemma) Let {an}°@, be a real sequence with the 
following property: there is an M > 0 such that |ay+...+an| <M for all positive 
integers N. Let {bn }S2, be a real sequence such that by > bo >... > bn... > 0. 
Then, for all N € Z*, | S>*_, anbp| < b1M. 


PRroor. Using (64), we have 


N N-1 
| So aabn| = | y An(dn = bn41) ae Anbn| 
n=1 n=1 


N-1 
< $2 (bn = bn41)|An| + bw |An| = bi |An| < 1M. 
n=1 


We will put Abel’s Lemma to use...but only much later on. For now though you 
might try to prove it without using the summation by parts variant (64): by doing 
so, you'll probably gain some appreciation that these formulas, though in some 
sense trivial, can be used in distinctly nontrivial ways. 


2. Easy Facts 
The following result collects some easy facts about convergence of infinite sequences. 


THEOREM 10.3. Let {an}, {bn}, {en} be real infinite sequences. 
a) If dn =C for alln — a constant sequence — then an > C. 
b) The limit of a convergent sequence is unique: if for L1, Lg € R we have an > Ly 
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and Gn > Lg, then Ly = Lo. 

Cc) If an > L and by, + M then: 

(i) For all CER, Ca, > CL. 

(it) an +bn 9 L+M. 

(itt) Qnbn > LM. 

(iv) If M 40, & > #.- 

d) If ay < by for all n, a, > L and b, > M, then Lb <M. 

e) Ifa <b are such that ay, € [a,b] for all n and a, — L, then L € [a,b]. 

f) (Three Sequence Principle) Suppose Cn = dn + bn. Then it is not possible for 
exactly two of the three sequences {an}, {bn}, {Cn} to be convergent: if any two are 
convergent, then so is the third. 

g) Suppose there exists N € Z* such that by = ay for alln > N. Then for any 
L € [-00, 00], an 9 LD SS bn OL. 


Most of these facts are qiute familiar, and the ones that may not be are routine. 
In fact, every part of Theorem 10.3 holds verbatim for functions of a continuous 
variable approaching infinity. Hence one method of proof would be to establish 
these for functions — or maintain that we have known these facts for a long time — 
and then apply the Sequence Interpolation Theorem. But be honest with yourself: 
for each part of Theorem 10.3 for which you have an iota of doubt as to how to 
prove, please take some time right now to write out a careful proof. 


We say that a sequence {a,,} is eventually constant if there is C € Rand N € Zt 
such that a, = C for all n > N. It is easy to see that if such a C’ exists then it 
is unique, and we call such a C’ the eventual value of the sequence. Of course 
an eventually constant sequence converges to its eventual value — e.g. by applying 
parts a) and g) of Theorem 10.3, but really this is almost obvous in any event. 


PROPOSITION 10.4. Let {an} be an infinite sequence with values in the integers 
Z. Then an is convergent iff it is eventually constant. 


Proor. As above, it is clear that an eventually constant sequence is convergent. 
Conversely, suppose a, > L € R. First we claim that L € Z. If not, the distance 
from L to the nearest integer is a positive number, say «. But since a, — L, there 
exists N € Z* such that for alln > N, la, —L| < e. But the interval (L —e¢, L+e) 
contains no integers: contradiction. 

Now take € = 1 in the definition of convergence: there is N € Z* such that for 
n > N, we have |a, — L| < 1, and since a, and L are both integers this implies 
Gd, = L. Thus the sequence is eventually constant with eventual value L. 


Proposition 10.4 goes a long way towards explaining why we have described a func- 
tion from Zt to R as semi-discrete. A function from Z* to Z* is “fully discrete”, 
and thus the limiting behavior of such functions is very limited. 


Exercise: A subset S C R is discrete if for all x € S, there is « > 0 such that the 
only element of S which lies in (x — €,4+ €) is x. 

a) Which of the following subsets of R are discrete? 

(i) A finite set. 

(ii) The integers Z. 

(iii) The rational numbers Q. 

(iv) The set {4 | n € Z*} of reciprocals of positive integers. 
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(v) The set {4 | n € Z*}U{O}. 

b) For a subset S C R, show that the following are equivalent: 

(i) S is discrete. 

(ii) Every convergent sequence {a,} with a, € S for all n is eventually constant. 


It is often convenient to consider sequences whose initial term is something other 
than 1. It is certainly no problem to entertain sequences starting at any integer 
No: informally, they look like 


QNo»ANotli++++ 
Formally, instead of a function from Zt to R, we have a function from {n € Z| n> 
No} to R. This mild generalization changes nothing we have said so far or will say 
later. We leave it to the reader to make her own peace with this. 


3. Characterizing Continuity 


THEOREM 10.5. Let I be an interval, let c be an interior point of I, and let 
f:IT-R be a function. The following are equivalent: 
(i) f is continuous at c. 
(ii) For every infinite sequence a, > c, we have f(an) > f(c). 


PRoor. (i) ==> (ii): This argument is very similar to the (easy!) proof that 
a composition of continuous functions is continuous. Namely, fix « > 0. Since f 
is continuous at c, there is 6 > 0 such that jr —c| <6 = |f(x) — f(d| <e. 
Moreover, since a, — c, there exists N € Zt such that n > N = > |a, —c| < 6. 
So ifn > N we have |a, — c| < 6 and thus |f(a,) — f(o)| <e. 
(ii) ==> (i): We prove the contrapositive: that is, we will suppose that f is not 
continuous at c and find a sequence a, — c but such that f(a,,) does not converge 
to f(c). If f is not continuous at c then there exists « > 0 such that for all 6 > 0, 
there is as with |as — c| < 6 and |f(as) — f(c)| > «. In particular, for n € Z* we 
may take 6 = 4 and choose a, with |an — cl < 4+ and |f(an) — f(c)| > €, and then 
indeed we have a, + c but f(an) does not approach f(c). 


Spivak gives a slightly more general result, the proof of which (essentially the same 
as the one given above) we leave as an exercise. 


THEOREM 10.6. Let I be an interval, c an interior point of I, and let f : 
I \ {c} > R be a function such that lim, f(x) = L. Let {a,}°2, be a real 
sequence such that for alln € Zt, an € I \ {c} and limps dn =c. Then 


lim f(a,) = LD. 


noo 


Exercise: Prove Theorem 10.6. 


4. Monotone Sequences 


One commonality between sequences a : Zt — R and non-discrete functions 
f : I — Ris that — since indeed both the domain and codomain are subsets of 
R — we have a natural order structure < and it makes sense to consider functions 
which systematically preserve — or systematically reverse! — this order structure. 
Thus the following definitions will be quite familiar. 


A sequence {a,,} is weakly increasing if for all m,n € Z*, m<n => Gm < Gn. 
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A sequence {a,,} is increasing if for all m,n € Z*,m <n => Gm < Gn. 

A sequence {a,} is weakly decreasing if for allm,n € Zt, m <n => Gm > an. 
A sequence {a,,} is decreasing if for all m,n € Z*, m<n => Gm > Gn. 

A sequence is monotone if it is weakly increasing or weakly decreasing. 


Remark: As before, we need to counsel the reader that the terminology here is 
not agreed upon: for instance, many use the term “strictly increasing” for se- 
quences with m <n = > dm < Gy. If you are ever in doubt whether someone 
meansm <n = > Am <Gno0rm<n = > Gm < Gn, there is a simple remedy: ask! 


Exercise: Let {an} be a sequence. 

a) Show that {a,,} is increasing iff an < an41 for alln € Z*. 

b) Formulate (and, if you feel it is worth your time, prove) analogues of part a) 
with increasing replaced by each of weakly increasing, decreasing, weakly decreasing. 


Exercise: For a real sequence {a,,}, show that the following are equivalent: 

(i) There is n € Zt such that either an < Qn41 > Qn42 (A-configuration) or 
Gn > An41 < Gn42 (V-configuration). 

(ii) The sequence {a,,} is not monotone. 


LEMMA 10.7. For a real sequence {an}, the following are equivalent: 
(i) {an} is both weakly increasing and weakly decreasing. 
(ii) {ay} is constant. 


Exercise: Prove it. 


LEMMA 10.8. (Reflection Principle) Let {a,} be a real sequence. Then: 
a) {an} is decreasing iff {—an} is increasing. 
b) {an} is weakly increasing iff {—an} is weakly increasing. 


Exercise: Prove it. 


Just as for functions of a continuous variable, Lemma 10.8 implies that whatever 
we can establish for increasing / weakly increasing sequences will carry over imme- 
diately to decreasing / weakly decreasing sequences (and conversely, of course). 


What a monotone sequence lacks is oscillation, and as for functions of a continuous 
variable, this implies a much simpler limiting behavior. Especially, if a, : Z+ > R 
is weakly increasing, there is really only one way for limn—sco an to fail to exist. 


To speak of this it is convenient to make another definition: since a sequence ay 
is really a function Zt — R, it makes sense to consider its image, i.e., the set of 
all real numbers of the form a, for some n € Z*. Strangely, there is not really a 
standard name for this: I vaguely recall it being called the “trace” of the sequence, 
but the term “trace” has many other, unrelated meanings in mathematics so this 
is really not optimal. We avoid the problem by giving the concept a very clunky 
name: we define the term set of {a,} to be A= {an €R| ne Zr}. 
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We are therefore able to speak of bounded sequences just as for bounded func- 
tions: i.e., in terms of the image...um, I mean the term set. 


A sequence a, : Z+ — R is bounded above if its term set is bounded above: 
that is, if there exists M € R such that a, < M for all n € Z*. Otherwise we say 
the sequence is unbounded above. Similarly, we say a, is bounded below if its 
term set is bounded below: that is, if there exists m € R such that m < a, for all 
n€ Z*. Otherewise we say the sequence is unbounded below. Finally, a sequenc 
is bounded if it is both bounded above and bounded below, and a sequence is 
unbounded if it is not bounded. 


PROPOSITION 10.9. Let {@,}°2, be a weakly increasing sequence. 
a) If the sequence converges to L € R, then L is the least upper bound of the term 
set A= {an |n € ZT}. 
b) Conversely, if the term set A has an upper bound L < c, then a, > L. 


PRooF. a) First we claim L = limn-yo0 dn is an upper bound for the term 
set A. Indeed, suppose not: then there is N € Z* with L < ay. But since the 
sequence is weakly increasing, this implies that for alln > N, L <ay <a,. Thus 
if we take € = ay — L, then for no n > N do we have |a, — L| < €, contradicting 
our assumption that a, — L. Second we claim L is the least upper bound. Indeed, 
suppose not: then there is L’ such that for alln € Z*, ay < L' < L. Lete = L—-L’. 
For no n do we have |a,, — L| < €, contradicting our asumption that a, — L. 

b) Let ¢ > 0. We need to show that for all but finitely many n € Z* we have 
—e< L—a, <e. Since L is the least upper bound of A, in particular L > a, for 
alln € Z*, so L—a, > 0 > —e. Next suppose that there are infinitely many terms 
Gn with LD —a, >, or L >a, +. But if this inequality holds for ifninitely many 
terms of the sequence, then because a, is increasing, it holds for all terms of the 
sequence, and this implies that L —¢ > a, for all n, so that DL —€ is a smaller upper 
bound for A than L, contradiction. 


Remark: In the previous result we have not used the completeness property of 
R, and thus it holds for sequences with values in the rationals Q (and where by 
converges we mean converges to a rational number!) or really in any ordered field. 
By combining this with the least upper bound axiom, we get a much stronger result. 


THEOREM 10.10. Let {a,}°2, be a weakly increasing real sequence. Let L € 
(—o0, co] be the least upper bound of the term set of A. Then an, — L. 


This is so important as to be worth spelling out very carefully. We get: 


THEOREM 10.11. (Monotone Sequence Theorem) a) Every bounded monotone 
real sequence is convergent. More precisely: 
b) Let {an} be weakly increasing. Then if {a,} is bounded above, it converges to its 
least upper bound, and if it is unbounded above it diverges to oo. 
c) Let {an} be weakly decreasing. Then if {an} is bounded below, it converges to its 
greatest lower bound, and it is unbounded below it diverges to —oo. 


In fact, in proving the Monotone Sequence Theorem we did not just invoke the 
completeness of the real field: we used its full force, in the following sense. 
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THEOREM 10.12. Let F be an ordered field in which every bounded monotone 
sequence converges. Then F' is Dedekind complete: every nonempty bounded above 
subset has a least upper bound. 


PROOF. Step 1: We CLAIM that F' is Archimedean. 
PROOF OF CLAIM: Suppose not: then the sequence x, = n is increasing and 
bounded above. Suppose that it were convergent, say to L € F. By Proposition 
10.9, L is the least upper bound of Z*. But this is absurd: if n < L for alln € Zt 
then n +1 < LF for all n € Zt and thus n < L—1 for alln € Zt, soL—lisa 
smaller upper bound for Z*. 

Step 2: Let S C R be nonempty and bounded above by Mo. 

CLAIM For all n € Z*, there exists yp, € 9 such that for any 7 € S, 2 < y+ 1. 
Proof of claim: Indeed, first choose any element z; of S. If for alla € S, a < z2,+ 1, 
then we may put yn = 21. Otherwise there exists z2 € S with z2 > 21 + 1. If for 
alae S,x2< z+ 1, then we may put yn = z2. Otherwise, there exists z3 € S 
with z3 > zg+ 4. If this process continues infinitely, we get a sequence with 
Zh > 21+ ae But by Step 1, F' is Archimedean, so that for sufficiently large k, 
zp > M, contradiction. Therefore the process musts terminate and we may take 
Yn = Zk for sufficiently large k. 

Now we define a sequence of upper bounds {M,,}°2, of S as follows: for alln € Z*, 
M, = min(Mn-1, yn + +). {M,,} is decreasing and bounded below by any element 
of S, so by hypothesis it converges, say to M, and by the reflected version of 
Proposition 10.9, M is the infimum of the set {M/,,}. Moreover M must be the 
supremum of S, since again by the Archimedean nature of the order, for any m < M, 
for sufficiently large n we have m+ 4 <M <M, < yn +2 and thus m < yp. 


n 


5. Subsequences 
A real infinite sequence {a,,} is, informally, an orderest list of real numbers: 
Q1,942,---,Q4n;--- 


A subsequence of {a,,}, is — again informally, for a little while; a precise definition 
will be coming soon — obtained by selecting some infinitely many terms of the 
sequence, e.g 

41,43, 96, 4100; 4703;---- 
(There is not meant to be a clearly evident pattern as to which terms of the se- 
quence we are choosing: the point is that there does not need to be.) To learn more 
about subsequences and what they are good for, let us look at some key examples. 


Example: Let a, = (—1)”, so the sequence is 

(65) —1,1,-1,1,... 

By selecting all the odd-numbered terms we get one subsequence: 

(66) —1,-1,-1,-1,... 

Similarly, by selecting all the even-numberd terms we get another subsequence: 
(67) Lp 1pi, dyes. 


There are other choices — many other choices. In fact, a real sequence can be ob- 
tained as a subsequence of {a,,} iff it takes values in {+1}. 
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But note that something very interesting happened in the passage from our original 
sequence to each of the first two subsequences. The original sequence (65) is not 
convergent, due to oscillation. However, the subsequence (66) is constant, hence 
convergent to it constant value —1. Similarly, the subsequence (67) converges to 
its constant value 1. 

Let’s recap: we began with a sequence (67) which did not converge due to oscil- 
lation. However, by choosing appropriate subsequences we were able to remove the 
oscillation, resulting — in this case, at least — in a convergent subsequence. (There 
are also lots of subsequences which are “inappropriate” for this purpose.) 


Example: Let a, =n, so the sequence is 
(68) 19, 3 Ae 


Here are some subsequences we can get: 


(69) Say eee 
(70) 2,4,6,8,... 
(71) 1,4,9,16,... 
(72) ee ee 


And so forth. Indeed, the subsequences of (68) are precisely the increasing se- 
quences with values in the positive integers. Note moreover that our sequence (68) 
fails to converge, but not due to oscillation. It is an increasing sequence which is 
not unbounded above, and thus it diverges to infinity. For that matter, so do the 
subsequences (69), (70), (71), (72), and a little thought suggests that every subse- 
quence will have this property. Thus, passing to a subsequence can cure divergence 
due to oscillation, but not divergence to infinity. 


Example (subsequences of a convergent sequence): 


We are now well-prepared for the formal definition. In fact, we practically saw 
it in the example above. Given a real sequence {a,}, we view it as a function 
ade: ZR, n+ ay. To obtain a subsequence we choose an increasing sequence 
ne: Z* 3Z+, kt ng and form the composite function 


aon: Zt OR, k++ an, - 


Less formally, we choose an increasing list ny < ng <... < nx of positive integers 
and use this to tell us which terms of the sequence to take, getting 


Gis Grins Gaigpe sf Ong ees 
Let’s formalize these observations about what passing to subsequences does for us. 
Exercise: Let n. : Z* — Zt be increasing. Show that for all k € Z+, nz > k. 


PROPOSITION 10.13. Let {an} be a real sequence, L € [—00, co], and suppose 
that a, — L. Then for any subsequence {an, }P2,, we have an, > L. 
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PrRoor. Case 1: Suppose that L € R. Since a, — L, for each € > 0 there 
exists N = N(e) € Z* such that for all n > N, |a, — L| < «. Then, by Exercise 
X.X, for all k > N we have n;, > N and thus lan, — L| < . 

Case 2: Suppose LZ = oo. Since an — oo, for each M € R there exists N = 
N(M) € Z* such that for alln > N, a, > M. Ifk > N, thenn, > k > N and 
thus an, > M. 

Case 3: We leave the case of L = —oo to the reader as a (not very challenging) 
exercise in modifying the argument of Case 2. 


PROPOSITION 10.14. Every subsequence of a monotone sequence is monotone. 
More precisely: 
a) If {a,} is weakly increasing, then every subsequence Gn, is weakly increasing. 
b) If {an} ts increasing, then every subsequence ay, is increasing. 
c) If {an} is weakly decreasing, then every subsequence Gn, is weakly decreasing. 
d) If {a,} is decreasing, then every subsequence an, 1s decreasing. 
e) If {an}, is constant, then every subsequence Gn, is constant. 


PROOF. a) If ki < ke, then by definition of a subsequence nz, < ng., and then 
by definition of weakly increasing, @n,, < n,,,- 
b),c),d),e) These may safely be left to the reader. 


COROLLARY 10.15. For a montone sequence {a,}, the following are equivalent: 
(i) {an} is convergent. 
(it) {an} is bounded. 
(itt) Every subsequence of {a,} is convergent. 
(iv) At least one subsequence of {a} is convergent. 


6. The Bolzano-Weierstrass Theorem For Sequences 


6.1. The Rising Sun Lemma. 


I learned of the following result from Mr. Evangelos Kobotis in late 1994, in my 
first quarter of college at the University of Chicago. Because of its elegance, gener- 
ality and usefulness, it has stayed with me through my entire adult mathematical 
career. It seems that this argument goes back to a short note of Newman and 
Parsons [NP88]. 


LEMMA 10.16. Every infinite sequence has a monotone subsequence. 


PRooF. Let us say that m € N is a peak of the sequence {a,,} if for all n > m, 
Gn < Gm. Suppose first that there are infinitely many peaks. Then any sequence 
of peaks forms a strictly decreasing subsequence, hence we have found a monotone 
subsequence. So suppose on the contrary that there are only finitely many peaks, 
and let N € N be such that there are no peaks n > N. Since nj = N is not a 
peak, there exists ng > n1 with an, > Gyn,. Similarly, since nz is not a peak, there 
exists 23 > ng with an, > dn,. Continuing in this way we construct an infinite (not 
necessarily strictly) increasing subsequence Gy,,4n,,---,4n,,---- Done! 


Exercise: Show that every infinite sequence admits a subsequence which is (i) in- 
creasing, (ii) decreasing, or (iii) constant. 


Remark: Although it is nothing to do with our development of infinite sequences, we 
cannot resist mentioning the following finite analogue of the Rising Sun Lemma. 
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THEOREM 10.17. (Erdés-Szekeres [ES35]) Let r,s € Z*. 
a) Consider a finite real sequence %1,...,£(r—1)(s—1)41 of length (r —1)(s—1) +1. 
(i) There is a weakly increasing subsequence of length r, or 
(it) There is a weakly decreasing subsequence of length s. 
b) The number (r — 1)(s — 1) +1 ts best possible, in the sense that there are real 
sequences of length (r —1)(s —1) without either a weakly increasing subsequence of 
length r or a weakly decreasing subsequence of length s. 


PROOF. a) ([Se59], [Er94]) Seeking a contradiction, we suppose every weakly 
increasing subsequence has length at most r—1 and every weakly decreasing subse- 
quence has length at most s—1. We define a function F': {1,...,(r—1)(s—1)+1} > 
{1,...,r —1} x {1,...,s —1} by F(y) = (;,d;), where i; is the length of the 
longest weakly increasing subsequence beginning with x; and d; is the length of 
the longest weakly decreasing subsequence beginning with x;. Since the domain of 
F has (r—1)(s—1)+1 elements and the codomain has (r —1)(s—1) elements, there 
must be j < k such that F(j) = F(k): thus i; = i, and d; = dy. But if x; < xp 
then adding x; to the beginning of the longest weakly increasing sequence starting 
with x, gives a longer weakly increasing sequence starting at x;, a contradiction; 
similarly, if x; > xp, then adding x; to the beginning of the longest weakly decreas- 
ing sequence starting with x, gives a longer weakly decreasing sequence starting at 
xj, a contradiction. b) Consider the length (r — 1)(s — 1) sequence 


rr—1,...,1,2r,2r—-1,...,r4+1,3r,3r—1,...,2r+1,...,9r,sr—1,...,(s—1)r41. 


6.2. Bolzano-Weierstrass for Sequences. 


THEOREM 10.18. (Bolzano-Weierstrass) Every bounded real sequence admits 
a convergent subsequence. 


ProoF. Let {an} be a sequence with |a,| <M for all n € Zt. By the Rising 
Sun Lemma, there is a monotone subsequence a, , which of course is also bounded: 
lan,| < M for all k € Z*+. By Corollary 10.15, the subsequence a, is convergent. 


Exercise: Show that for an ordered field F’, the following are equivalent: 
(i) Every bounded monotone sequence converges.! 
(ii) Every bounded sequence admits a convergent subsequence. 


6.3. Supplements to Bolzano- Weierstrass. 


THEOREM 10.19. a) A real sequence which is unbounded above admits a sub- 
sequence diverging to co. 
b) A real sequence which is unbounded below admits a subsequence diverging to —oo. 


Proor. We will prove part a) and leave the task of adapting the argument to 
prove part b) to the reader. Let {x,,} be a real sequence which is unbounded above. 
Then for every M € R, there exists at least one n such that x, > M. Let n, be the 
least positive integer such that x,,, > 1. Let ng be the least positive integer such 
that x, > max(%,,,2). And so forth: having defined nz, let nx41 be the least 
positive integer such that zp,,, > max(¢p,,k +1). Then limp_,oo &n,, = 00. 


Recall that we have already shown that this is equivalent to Dedekind completeness. 


210 10. INFINITE SEQUENCES 
6.4. Applications of Bolzano- Weierstrass. 


We will now give two quite different proofs of two of the three Interval Theorems. 


THEOREM 10.20. (Extreme Value Theorem Again) Let f : [a,b] > R be con- 
tinuous. Then f is bounded and attains its minimum and maximum values. 


PROOF. Seeking a contradiction we suppose f is unbounded: then, for each 
n € Zt, there exists x, € [a,b] with |f(r,)| > n. The sequence {z,} takes val- 
ues in [a,b] so is bounded, so by Bolzano-Weierstrass there is a subsequence Zp, 
such that z,, — L € R. Now, on the one hand, since f is continuous we have by 
Theorem 10.5 that f(¢n,) > f(Z) € R. On the other hand, we have for all k that 
f(Xn,)| > &, so the sequence f(x,,) is divergent: contradiction! 

We do not have a new argument for the attainment of the minimum and max- 
imum values, but the original argument was sufficiently short and sweet that we 
don’t mind repeating it here: let M be the supremum of the set of f(a) for x € [a, bl, 
so by what we just proved, M < oo. If M is not attained then the function f(x«)—M 
is continuous and nowhere zero, so the function g : [a,b] — R by g(x) = F@)-M 


is continuous on [a,b], hence bounded. But this is impossible: by definition of M, 
f(a) takes values arbitrarily close to M, so f(a) — M takes values arbitrarily close 
to zero and thus |g(x)| take arbitrarily large values. By reflection, f attains its 
minimum as well. 


THEOREM 10.21. (Uniform Continuity Theorem Again) Let f : [a,b] > R be 
continuous. Then f is uniformly continuous. 


PRooF. Suppose not: then there exists « > 0 such that for all n € Z* we have 
In, Yn € [a,b] with |» —yn| < + but |f(an) — f(Yn)| > €. By Bolzano-Weierstrass, 
after passing to a subsequence we may assume 2, is convergent, say to L, Since 
In, — Yn, — 0, by the Three Sequence Principle y,, is also convergent to L. But 
then f(%n,) + f(L) and f(yn,) > f(L), and since the sequences f(tp,) and f (Yn, ) 
have the same limit, we can make | f(an,)—f(Yn,)| < |f(an,) -L—-(f (unm, )) -L£)| < 
|f(an,) — L| + |f(Yn,) — £| arbitrarily small by taking n sufficiently large. In 
particular, for sufficiently large n we have |f(an,) — f(Yn,)| < €: contradiction! 


Remark: Although I am very fond of the Real Induction proof of the Uniform Con- 
tinuity Theorem, I must admit this argument seems shorter and easier. Very often 
in mathematics if one takes the time to develop additional, seemingly unrelated 
technology — in this case, the theory of infinite sequences — one is richly rewarded 
with the ability to come up with shorter, simpler (but more “high tech”) proofs. 


6.5. Bolzano-Weierstrass for Subsets Revisited. 


Recall that earlier we proved a different result which we also called the Bolzano- 
Weierstrass Theorem: it concerned the notion of a limit point of a subset X C R: a 
point a € Risa limit point for X if for alle > 0, there is x € R with 0 < |a—2| <e. 
It was an easy observation that any finite subset of R has no limit points. We proved 
that the converse holds for bounded sets: 


THEOREM 10.22. (Bolzano-Weierstrass for Subsets) An infinite subset X of a 
closed, bounded interval [a,b] has a limit point. 
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Our task now is to explain why we have two different results called the Bolzano- 
Weierstrass Theorem. In fact they are really equivalent results, which means 
roughly that is much easier to deduce each from the other than it is to prove 
either one. Indeed: 


Assume the Bolzano-Weierstrass Theorem for sequences, and let X be an in- 
finite subset of [a,b]. Since X is infinite, we can choose a sequence {x,,}°2, of 
distinct elements of X. Since xz, € X C [a,b], we have a < x, < b for all n; 
in particular, {,,} is bounded, so by Bolzano-Weierstrass for sequences there is a 
subsequence £,, converging to some L € [a,b]. We claim that L is a limit point for 
X: indeed, for any € > 0, there is K € Z* such that for all k > K, |rp, — L| < ¢: 
since the terms are distinct, at most one of them is equal to LZ and thus the interval 
(L —e¢,£ + €) contains infinitely many elements of X. 

Assume the Bolzano- Weierstrass Theorem for subsets, and let {x,, } be a bounded 
sequence: thus there are a,b € R witha < rp < b for alln € Z*. Let X = {tn |ne€ 
Z*} be the term set of the sequence. If X is finite, then the sequence has a con- 
stant (hence convergent) subsequence. Otherwise X is infinite and we may apply 
Bolzano-Weierstrass for subsets to get a limit point L of X. This implies: there is 
ny € Z* such that |xz,, — L| < 1; having chosen such an nj, there is n2 € Zt such 
that ng > ny and |r, — L| < 4: continuing in this way we build a subsequence 
{&n, } such that for all k € Z*, |z,, —L| < 4, and thus zp, — L. 


7. Partial Limits; Limits Superior and Inferior 


7.1. Partial Limits. 


For a real sequence {a,}, we say that an extended real number L € [—o0, 00] 
is a partial limit of {a,} if there exists a subsequence ay, such that an, > L. 


LEMMA 10.23. Let {an} be a real sequence. Suppose that L is a partial limit of 
some subsequence of {a,}. Then L is also a partial limit of {ay}. 


Exercise: Prove Lemma 10.23. (Hint: this comes down to the fact that a subse- 
quence of a subsequence is itself a subsequence.) 


THEOREM 10.24. Let {a,,} be a real sequence. 
a) {an} has at least one partial limit L € [—co, ox]. 
b) The sequence {a,} is convergent iff it has exactly one partial limit L and L is 
finite, t.e., LD A too. 
C) An > © iff co is the only partial limit. 
d) an — —00 iff —oo is the only partial limit. 


Proor. a) If {an} is bounded, then by Bolzano-Weierstrass there is a finite 
partial limit L. If {a,} is unbounded above, then by Theorem 10.19a), 00 is a 
partial limit. It {a,} is unbounded below, then by Theorem 10.19b) —oo is a 
partial limit. Every sequence is either bounded, unbounded above or unbounded 
below (and the last two are not mutually exclusive), so there is always at least one 
partial limit. 

b) Suppose that L € R is the unique partial limit of {a,}. We wish to show that 
Gn — L. First observe that by Theorem 10.19, {a,} must be bounded above and 
below, for otherwise it would have an infinite partial limit. So choose M € R such 
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that |a,| < M for all n. 

Now suppose that a, does not converge to L: then there exists € > 0 such that 
it is not the case that there exists N € N such that |a, — L| < € for alln > N. 
What this means is that there are infinitely many values of n such that |an —L| > e. 
Moreover, since |a, — L| > € means either —M <a, < L—eor L+e<a, <M, 
there must in fact be in infinite subset S C N such that either for all n € S we have 
ay € |[—M, L — | or for all n € S we have a, € [L + €, M]. 

Let us treat the former case. The reader who understands the argument will 
have no trouble adapting to the latter case. Writing the elements of S in increasing 
order as 21, 72,...,~, we have shown that there exists a subsequence {a,,, } all of 
whose terms lie in the closed interval [—M, L — «]. Applying Bolzano-Weierstrass 
to this subsequence, we get a subsubsequence (!) @n,, which converges to some L’. 
We note right away that a subsubsequence of a, is also a subsequence of a,: we 
still have an infinite subset of N whose elements are being taken in increasing order. 
Moreover, since every term of ay, is bounded above by L — e, its limit L’ must 
satisfy L’ < L—e. But then L’ 4 L so the sequence has a second partial limit L’: 
contradiction. 

c) Suppose a, — oo. Then also every subsequence diverges to +00, so +00 is 
a partial limit and there are no other partial limits. We will prove the converse 
via its contrapositive (the inverse): suppose that a, does not diverge to oo. Then 
there exists M € R and infinitely many n € Z* such that a, < M, and from this 
it follows that there is a subsequence {a,,} which is bounded above by M. This 
subsequence does not have +oo as a partial limit, hence by part a) it has some 
partial limit LD < oo. By Lemma 10.23, L is also a partial limit of the original 
sequence, so it is not the case that +00 is the only partial limit of {a,}. 

d) Left to the reader to prove, by adapting the argument of part c) or otherwise. 


Exercise: Let {x,,} be a real sequence. Suppose that: 

(i) Any two convergent subsequences converge to the same limit. 

(ii) {x,} is bounded. 

Show that {z,} is convergent. (Suggestion: Combine Theorem 10.24b) with the 
Bolzano-Weierstrass Theorem.) 


Exercise: Let {x,} be a real sequence, and let a < b be extended real numbers. 
Suppose that there exists N € Z+ such that for alln > N,a< 2x, < b. Show that 
{an} has a partial limit LD witha << L <b. 


7.2. The Limit Supremum and Limit Infimum. 
For a real sequence {a,,}, let £ be the set of all partial limits of {a, }. 


We define the limit supremum I of a real sequence to be the least upper bound 
of the set of all partial limits of the sequence. 


THEOREM 10.25. For any real sequence {an} , L is a partial limit of the se- 
quence and is thus the largest partial limit. 


PROOF. Case 1: The sequence is unbounded above. Then +oo is a partial 
limit, so L = +00 is a partial limit. 
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Case 2: The sequence diverges to —oo. Then —oo is the only partial limit and 
thus L = —oo is the largest partial limit. 


Case 3: The sequence is bounded above and does not diverge to —oo. Then it 
has a finite partial L (it may or may not also have —oo as a partial limit), so 
L € (—oo, 00). We need to find a subsequence converging to L. 

For each k € Z*, LT — i < L, so there exists a subsequence converging to some 
ES i= i: In particular, there exists n, such that an, > Gee i: It follows from 
these inequalities that the subsequence a,, cannot have any partial limit which 
is less than L; moreover, by the definition of L = sup £ the subsequence cannot 
have any partial limit which is strictly greater than L: therefore by the process of 
elimination we must have a, — L. 


Similarly we define the limit infimum L of a real sequence to be the infimum of the 
set of all partial limits. By reflection, the proof of Theorem 10.25 shows that L is a 
partial limit of the sequence, i.e., there exists a subsequence a,,, such that a,, — L. 


Here is a very useful characterization of the limit supremum of a sequence {a,, } it is 
the unique extended real number L such that for any M > L, {n € Zt | an > M} 
is finite, and such that for any m < L, {n € Zt | a, > m} is infinite. 


Exercise: 
a) Prove the above characterization of the limit supremum. 
b) State and prove an analogous characterization of the limit infimum. 


PROPOSITION 10.26. For any real sequence an, we have 
(73) T= lim supa, 


and 


ae Ee Ie Bn 
Because of these identities it is traditional to write limsupa, in place of L and 
liminf a, in place of L. 


Proor. As usual, we will prove the statements involving the limit supremum 
and leave the analogous case of the limit infimum to the reader. 

Our first order of business is to show that lim,_,.. Supys,, Gp eXists as an ex- 
tended real number. To see this, define b, = sup, s,,@z. The key observation is 
that {b,,} is decreasing. Indeed, when we pass from a set of extended real numbers 
to a subset, its supremum either stays the same or decreases. By Theorem 10.21, 
bp — L’ € [—00, ov}. 

Now we will show that ZT = L’ using the characterization of the limit supre- 
mum stated above. First suppose M > L’. Then there exists n € Z* such that 
SUPps, dk < M. Thus there are only finitely many terms of the sequence which are 
at least M, so M > L. It follows that L’ > L. 

On the other hand, suppose m < L’. Then there are infinitely many n € Z* 
such that m < an and hence m < L. It follows that LE < L’, and thus L = L’. 


The merit of these considerations is the following: if a sequence converges, we have a 
number to describe its limiting behavior, namely its limit L. If a sequence diverges 
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to too, again we have an “extended real number” that we can use to describe 


its limiting behavior. But a sequence can be more complicated than this: it may 
be highly oscillatory and therefore its limiting behavior may be hard to describe. 
However, to every sequence we have now associated two numbers: the limit infimum 
L and the limit supremum LF, such that 


-o <L<LD<+0. 


For many purposes — e.g. for making upper estimates — we can use the limit 
supremum L in the same way that we would use the limit L if the sequence were 
convergent (or divergent to too). Since L exists for any sequence, this is very 
powerful and useful. Similarly for L. 


CoROLLARY 10.27. A real sequence {an} is convergent iff L = L € (—ox, 00). 


Exercise: Prove Corollary 10.27. 


8. Cauchy Sequences 


8.1. Motivating Cauchy sequences. 


Let us look back at our definition of a convergent sequence: a real sequence {a,,} 
is convergent if there is some L € R such that {a,} converges to L. 

There is something slightly curious about this definition: do you see it? It is 
this: we are defining the notion “convergent” in terms of the notion “convergent 
to L”. One might think that things should go the other way around: shouldn’t the 
concept of convergene should be somehow logically prior than “convergence to L”? 
Anyway, this is the way we tend to think of things in practice: if one is given an 
infinite sequence, first we want to know whether it is convergent or divergent, and 
then if we know it is convergent we can ask the more refined question: “To which 
real number L does our convergent sequence converge?” 


This is a good opportunity to think back on our other notions of convergence. 

If you think about it, our definition of “the limit of f(a) as x > c” also has this 
feature: what we actually defined was lim,_,, f(a) = L. For continuity the issue is 
not pressing because we know that we want the limit to be f(c).? The concept of 
uniform continuity really does not mention a specific limiting value, i-e., a function 
is not “uniformly continuous to L atc’. 

Since derivatives are defined in terms of limits (and not in terms of continuity!) 
they have the same problem: to show that f is differentiable at c we have to show 
that the limiting slope of the secant line is some specific real number. 

For integrals, the plot thickens. Our first, awkward, definition involved a fixed 
real number J. Then we gave a modified definition in terms of two quantities a yi 


and ie f which exist for any function, and our definition of integrability was that 
these two quantities are finite and equal. This is sort of in between in that the defi- 
nition includes a particular limiting value but we don’t have to “find” it: we have to 
show that an underestimate and an overestimate are actually the same. Moreover 
we have Darboux’s criterion, which remedies the issue completely: this gives us a 
procedure to show that a function is integrable without requiring any knowledge on 


21 think this partially explains why the concept of continuity is simpler than that of a limit. 
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what number the integral should be. And that’s what made Darboux’s criterion 
so useful: we used it to show that every continuous function and every monotone 
function is integrable, but of course without having to “find” in any sense the value 
of the integral. (This inexplicitness is not entirely a good thing, and the main point 
of our discussion of Riemann sums was to make the convergence more explicit.) 


Upshot: it would be nice to have some way of expressing/proving that a sequence 
is convergent which doesn’t have the limit of the sequence built into it. This is 
exactly what Cauchy sequences are for. 


8.2. Cauchy sequences. 


A sequence {a,}°2, in is Cauchy if for all « > 0, there exists N € Z* such 
that for all m,n > N, |am — an| <e. 


Here are some elementary properties of Cauchy sequences.? 


LEMMA 10.28. A subsequence of a Cauchy sequence is Cauchy. 


PrRooF. Left to the reader as an exercise. 


PROPOSITION 10.29. A convergent sequence is Cauchy. 


PRoor. Suppose a, — L. Then there exists N € Z* such that for all n > N, 
|a, — L| < §. Thus for all m,n > N we have 


lan — Om| = |(@n — L) — (dm — £)| < lan — E+ lam — L] < 5+ 5 =e 


PROPOSITION 10.30. A Cauchy sequence is bounded. 


Proor. Let {a,} be a Cauchy sequence. There exists N € Z* such that for 
all m,n > N, |@m — Gn| < 1. Therefore, taking m = N we get that for alln > N, 
lan — an| < 1, so |an| < Jaw| + 1 = Mi, say. Moreover put Mz = maxi<n<n |an 
and M = max(M,, M2). Then for all n € Z*, |a,| < M. 


PROPOSITION 10.31. A Cauchy sequence which admits a convergent subse- 
quence is itself convergent. 


ProoF. Let {a,} be a Cauchy sequence and suppose there is a subsequence 
Gn, converging to L € F. We claim that a, converges to L. Fix any € > 0. 
Choose N; € Z* such that for all m,n > Ny we have |a@n — am| = |@m — an| < - 
Further, choose Nz € Zt such that for all k > Nz we have |an, — L| < §, and put 
N = max(N,, No). Then ny > N and N > Na, so 


€ 
Jan — L| = |(@n — Gay) — (4ny ~ £)| S lan — Gny| + |@nw —L)< 5 +5 =& 


The above proofs did not use completeness, and thus the results hold in any ordered 
field. In contrast, the next result does crucially use the Dedekind completeness of 
the real numbers, in the form of the Bolzano-Weierstrass Theorem. 


3You might notice that they are reminiscent of the elementary properties of monotone se- 
quences we proved above. But I myself noticed this only quite recently — more than 15 years 
after I first learned about monotone sequences and Cauchy sequences. I wish I had a deeper 
understanding of the underlying source — if any! — of these commonalities. 
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THEOREM 10.32. (Cauchy Criterion) Any real Cauchy sequence is convergent. 


Proor. Let {a,,} be a real Cauchy sequence. By Proposition 10.30, {a,} is 
bounded. By Bolzano-Weierstrass there exists a convergent subequence. Finally, 
by Proposition 10.31, this implies that {a} is convergent. 


It can be further shown that an Archimedean ordered field F in which every 
Cauchy sequence is convergent must be Dedekind complete. However, there are 
non-Archimedean — and thus necessarily not Dedekind complete — ordered fields 
in which every Cauchy sequence converges. (In fact there are non-Archimedean 
ordered fields in which the only Cauchy sequences are the eventually constant se- 
quences!) But we had better not get into such matters here. 


Exercise: Is there a “Cauchy criterion” for a function to be differentiable at a 
point? (Hint: yes. See [Ma56].) 


9. Geometric Sequences and Series 


A geometric sequence is a sequence {x,,}?2, of real numbers such that there is 
a fixed real number r with 2,41 = rv,. We call r the geometric ratio since, if 
for alln € Zt x, #0 we have ++ =r, 


THEOREM 10.33. (Geometric Sequences) Let {x} be a geometric sequence with 
geometric ratio r and xp #0. 
a) We have &, = ror”. 
b) If |r| <1, then tp > 0. 
c) Ifr =1, then tn > Xo. 
d) If r = —1, then the sequence is %9,—%0,X0,—Xo,..., which diverges. 
e) If |r| > 1, then |x| — oo. 


PRooF. a) A simple induction argument which is left to the reader. 
Suppose z, — L € [-—co, co]. Since try. = Tan, L = limpsotnzi = 
limy_soo T&n, = TL. The solutions to L = rL for r € R, L € [—ov, «] are: (1, L) for 
any L, (r,0) for any r, (r,0o) for positive r and (r, —oo) for negative r. Now: 
b) If |r| < 1, then |@n4i] = |r|lan| < |tn|, so {|an|} is decreasing and bounded 
below by 0. By the Montone Sequence Lemma, |z,,| converges to a finite, non- 
negative number L, and by the above analysis L = 0. Since |x,,| > 0, t, > 0. 
c), d) These are immediate and are just recorded for easy reference. 
e) If |r| > 1, then |ay41| = |r||an| > |vn|, so the sequence {|z,,|} is increasing and 
bounded below by |zo|. By the Monotone Sequence Lemma, |z,,| > L € (|xo|, co], 
and by the above analysis we must have L = oo. 


For x9,7r € R, we define the finite geometric series 
(75) Sy, =o +xXor+...+20r”. 
We claim that — quite luckily! — we are able to obtain a simple closed-form expres- 
sion for S,. We may dispose of the case r = 1: then S,, = (n+1). When r £1, we 
use a very standard trick: multiplying (76) by r gives 
(76) rSn = «ort... zor” +2or"™!, 
and subtracting (76) from (75) gives 
(l—-r)S_, =2a9(1—r?*?. 
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Since r # 1, this yields 
n 
i prt 
= Kee 
(77) c=) cor = =a 
k=0 
Having this closed form enables us to determine easily whether the sequence {S,,} 
converges, and if so, to what. 
THEOREM 10.34. (The Geometric Series) For xo,r € R, let 
n 
fol =e seo” =X +xXor+...+2or”. 


k=0 
1 


a) If |r| <1, then limpoo Sn = t=: 
b) If |r| > 1, then {S,} diverges. 
Exercise: Use Theorem 10.33 to prove Theorem 10.34. 
10. Contraction Mappings Revisited 

The material of this section owes a debt to a paper of K. Conrad [CdC]. 

10.1. Preliminaries. 
Recall that we last studied contractive mappings, fixed points and sequences of 
iterates in the context of Newton’s method. Now, towards the end of a long chap- 


ter on infinite sequences, we have more tools at our disposal, so it is reasonable to 
take another look at this important circle of ideas. Let’s take it again from the top: 


A function f : I > R is contractive, or a contraction mapping, if there is 
a constant 0 < C' < 1 such that: 


(78) Va,y €I,|f(x) — f(y)| < Cla — yl. 


Recall that f : I > R is Lipschitz if (78) holds for some C > 0. If so, for 
any € > 0, we may — independently of 2 — choose 6 < &, and then la —y| < 
6 => |f(«) — f(y)| < Cé < «. So a Lipschitz function is uniformly continuous. A 
contraction mapping is a Lipchitz mapping with Lipschitz constant C < 1. 


A function f : J + R is weakly contractive if: 


Ve AY ET, |f(x) — fy) <|e—yl- 
A function f : J + Ris a short map if 1 is a Lipschitz constant for f, i.e., 


Veuve l, fin —fws esol 


Thus contractive implies weakly contractive implies short map. 


Exercise: Let f : J > I be differentiable. 

a) Show that if | f’(2)| < C for all x € I, then C is a Lipschitz constant for f. 

b) Deduce: if there is C € (0,1) with |f’(x)| < C for all x € I, f is contractive. 

c) State and prove analogous sufficient conditions in terms of bounds on f’ for f 
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to be weakly contractive (resp. a short map). 


Let X and Y be sets, and let f : X — Y be a function (or, synonymously, a 
“map” or “mapping”). An element L € X is a fixed point for f if f(L) = L. The 
study of fixed points of mappings is highly important throughout mathematics. 

A fixed point L of f is, in particular, an element of both X and Y. Perhaps 
the sets X and Y have no elements in common: then of course no function between 
them can have a fixed point: e.g., looking for fixed points for mappings from the set 
X of real numbers to the set Y = {Barack Obama, Lady Gaga, William Faulkner} 
would be an exercise in futility. 

To alleviate this problem, we often restrict attention to mappings f : X > X. 
Further, such a mapping can be iterated: that is, we can take the output and 
plug it back in as the input: « € X 4 f(x) 4 f(f(x)). And we can keep going. 
Formally, starting from any initial value x) € X, we get a sequence of iterates 
{tn }p2-9 where for all n € N, fn41 = f(an). (Recall that Newton’s method in- 


volved consideration of sequences of iterates of the function T(x) = x — fe :) 


And now, back to earth: the methods of calculus allow us to analyze functions 
defined on subsets, especially intervals, of the real numbers. So suppose we have 
an interval J C R and a function f : J > J. Here is what we want to know. 


QUESTION 10.35. Let f: I > TI be a function. 
a) Does f have at least one fixed point? 
b) Does f have exactly one fixed point? 
c) If f has fixed points, how can we explicitly find them, at least approximately? 


10.2. Some Easy Results. 


Our first result is a very modest generalization of Lemma 7.7. 
THEOREM 10.36. A weakly contractive function has at most one fixed point. 


PRooF. Suppose there are distinct [1,22 € I such that f(L1) = Ly and 
f(L2) = Lz. Then |f(£1) — f(£2)| = |L£1 — Le|, contradiction. 


Exercise: Show that a short map f : J > I can have more than one fixed point. 


LEMMA 10.37. Let f : I > I be a continuous function. Let xo € I and let 
o,t1 = f(x1),%2 = f(x2),... be the sequence of iterates of xo under f. Suppose 
that t, converges to some LEI. Then L is a fixed point of f. 


PROOF. Since f is continuous and x, > L, f(x,) > f(L). But since f(¢,) = 
In+41, the sequence {f(%n)} = {an+1} is just a reindexed version of the sequence 
{tn}, so it must have the same limit: f(Z) = L. 


Exercise:* Consider the function f : (0, 1] > (0,1) given by f(x) = §. 
a) Show that f is contractive. 
b) Show that f has no fixed point on (0, 1]. 


Exercise: Consider the function f : R > R given by f(a) =x +1. 


4This exercise and the next are easy bordering on insulting, but to properly appreciate later 
results it will be useful to have these examples on record. 


10. CONTRACTION MAPPINGS REVISITED 219 


a) Show that f is a short map, hence continuous. 
b) Show that f has no fixed point on R. 


Exercise: Consider the function f : R > R given by f(x) = Vx? +1. 
a) Show that f is weakly contractive. 

b) Show that f has no fixed point on R. 

c) Show that f is not contractive. 


When f : J — I has a fixed point L, we would like to be able to find LD as the 
limit of a sequence of iterates. And we can: if zo = L, then f(x%0) = L, so the 
sequence of iterates is constantly LZ. But this observation is not very useful in find- 
ing L! The next, slightly less trivial, observation is that a convergent sequence of 
iterates of a continuous function yields a fixed point. To state this more precisely 
we introduce some further terminology. 


We say L € I is an attracting point for f if for every x) € I, the sequence 
of iterates xo, f (xo), f(f(@o)),... converges to L. We say f is attractive if it has 
an attracting point. 


LEMMA 10.38. Suppose f : I — I is an attractive function. 
a) The attracting point L is unique. 
b) f has at most one fixed point. 
c) If f is continuous, then the unique attracting point is the unique fixed point. 


PROOF. a) Suppose L; # Lg are attractive points. Starting at any initial point 
xo € L, the sequence of iterates of x) under f must converge to both Ly, and Lg. 
Since a sequence can have at most one limit, this is a contradiction. 

b) If L is a fixed point for f, then the sequence of iterates of LD under f is constantly 
Land thus converges to L. So if we had at least two fixed points, different sequences 
of iterates would have at least two different limits, contradicting attractivity. 
c) This follows immediately from the first two parts and from Lemma 10.37. 


Thus attractive maps are precisely the maps with a unique fixed point which can 
be found as the limit of a sequence of iterates starting from any point xo in the 
domain. So the fixed point of an attractive map is easy to find: we just start any- 
where, iterate, and we get sucked into it. 


Exercise: Consider the function f : [0,1] — [0,1] such that 


5,0<a<l 
—) @ = 
oe eae 
a) Show that 0 is an attractive point of f but not a fixed point. 
b) Why does this not contradict Lemma 10.38? 


Remark: Note that, although our terminology suppresses this, whether D is an 
attracting fixed point for f depends very much on the interval of definition J. In 
the literature, the term “attracting point” is often used for a weaker, “local prop- 
erty” which is studied in the following exercise. 


Exercise: Let f : I + I be a function. A point LZ € J is a locally attracting 
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point if there exists 6 > 0 such that f maps [LZ — 6,L + 6] into itself and the 
restriction of f to [L — 6, £ + 6] has L as an attracting point. 

Now let f : 1 + I be C!, and let L € I° be a fixed point of f. 
a) Show that if |f’(Z)| < 1, then L is a locally attracting point for f. 
b) Show that if |f’(Z)| > 1, then L is not a locally attracting point for f. 
c) Exhibit f : [-1,1] — R such that L = 0 is a fixed point, f’(0) = 1, and 0 isa 
locally attracting point for f. 
d) Exhibit f : [-1,1] > R such that L = 0 is a fixed point, f’(0) = 1 and 0 is not 
a locally attracting point for f. 


10.3. The Contraction Mapping Theorem. 


THEOREM 10.39. (Contraction Mapping Theorem) Let I Cc R be a closed in- 
terval, and let f : I + I be contractive with constant C € (0,1). 
a) Then f is attractive: there is a unique fixed point L, and for all xo € I, the 
sequence {x,,} of iterates of f under xo converges to L. 
b) Explicitly, for allay € I and alln EN, 


|zo — f(%o)| 
79 n-L\< Co”. 
(79) jm — 2) < (OZ 
PROOF. Step 0: By Theorem 10.36, f has at most one fixed point. 


Step 1: Let xo € J, fix « > 0, let N be a large positive integer to be chosen (rather 
sooner than) later, let n > N and let k > 0. Then 


\Sn+k —2n| < |fnae — 2nik i] + |an+r 1—Xn+k alt... + |any1 — 2n| 
OT ete) Olay ral ee Cole a 
= |g = 2516" CFC Fue CY = |e ~ 2910" (7S) z (SS) er 
1-C Lae 


Since |C| < 1, C" > 0, so we may choose N such that for alln > N and allk EN, 


|v1—2o| 


1G ) C" <.e. So {a} is Cauchy and hence, by Theorem 10.32, convergent. 
Step 2: By Step 1 and Lemma 10.37, for each x € I the sequence of iterates of x 
under f converges to a fixed point. By Step 0, f has at most one fixed point. So 
there must be a unique fixed point L € I, which is the limit of every sequence of 
iterates: i.e., L is an attracting point for f. This completes the proof of part a). 
Step 3: We leave the derivation of (79) as an exercise for the reader. 


Exercise: Complete the proof of Theorem 10.39 by establishing (79). (Suggestion: 
adapt the argument of Step 1 of the proof.) 


Thus the Contraction Mapping Theorem asserts in particular that a contractive 
mapping on a closed interval is attractive. That’s a fine result in and of itself. 
But we have more, namely the explicit estimate (79). It guarantees that starting 
at any point x € J the sequence of iterates converges to the attracting point L 
exponentially fast, and is so explicit that it immediately gives an efficient algorithm 
for approximating L to any desired accuracy. Now that’s a theorem! 


Exercise: a) Show that there is a unique real number « such that cos x = a. 
b) Explain how to use (say) a handheld calculator to approximate x to as many 
decimal places of accuracy as your calculator carries. 
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Exercise: Let f : [a,b) — [a,b) be a contractive map with constant C' € (0,1). 

a) Show that lim, _,,- f(a) exists (as a real number!), and that by defining f at 
b to be this limit, the extended function f : [a,b] > R remains contractive with 
constant C.° 

b) Use part a) to extend the Contraction Mapping Principle to contractions f : 
[a,b) — [a,b), with the proviso that the unique fixed point may be b. 

c) Give an example of a contraction mapping on [a, b) with fixed point b. 

d) State and prove a version of the Contraction Mapping Principle valid for an 
arbitrary interval. 


Exercise: Let f : I + I be a function. For n € Z*, let f°" = fo...of be 
the nth iterate of f. Suppose that for some N € Z+, f°% is contractive. 

a) Show that any fixed point of f is a fixed point of f°’. 

b) Show that f°’ has a unique fixed point L € I, and deduce that f has at most 
one fixed point in I. 

Show that f(L) is also a fixed point for f°’, and deduce that f(L) = L: thus f 
as a unique fixed point in J. 

) Show that in fact LZ is an attracting point for f. 

Consider the function f : [0,1] — [0,1] by 


se) ={ 0,0<2re€ [0,1] 


c 
h 
d 
e 


1, x € (1, 2}. 
Show that f is not a contraction but fo f is a contraction. 


10.4. Further Attraction Theorems. 


Let J be an interval, and let f : I — J be continuous. Recall that L € I is 
attracting for f if for all vo € J, the sequence of iterates of xq under f converges to 
L; above, we showed that if L is attracting for f then L is the unique fixed point 
of f. Let us say sup/ is an attracting point for f if for all rq € J, the sequence 
of iterates of x under f approaches sup/ (sup J = oo iff J is unbounded above). 
Similarly, we say inf J is an attracting point for f if for all xo € J, the sequence of 
iterates of zo under f approaches inf J (inf J = —oo iff J is unbounded below). 


THEOREM 10.40. Let IC R be an interval, and let f : I > I be continuous. 
a) At least one of the following holds: 
(i) f has a fixed point. 
(it) sup I is an attracting point for f. 
(itt) inf I is an attracting point for f. 
b) If I = [a,b] then f has a fixed point. 


PROOF. a) Define g: J > R by g(x) = f(x) —«. A fixed point of f is precisely 
a root of g, so to prove part a) we may assume that g has no roots on J and show 
that either (ii) or (iii) holds. If the continuous function g has no roots then either 
(1) f(x) > @ for all « € I or (II) f(x) < a for all x € I. We will show (1) => (ii); 
the very similar proof that (II) => (iii) is left to the reader. 

Suppose f(a) > x for all « € J. Then, for any x € J, the sequence of iterates 


5This is the hardest part of the problem. The result on extension will be much easier after 
you have read the next section, which treats similar but more general problems. You may wish to 
assume this part for now and come back to it later. 
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is increasing. This sequence cannot converge to any element L of J, for by Lemma 
10.37, L would then be a fixed point of f, contradiction. So x, must approach 
sup I. 

b) Let I = [a,b] and let zo € I. If there were no fixed point, then by part a) the 
sequence of iterates of x) under f would approach either sup = 6 or inf I = a. 
But both are elements of J, so by Lemma 10.37 either a or b is a fixed point. 


THEOREM 10.41. Let I C R be an interval, and let f : I + I be weakly 
contractive. Then f has an attracting point in [inf I, sup J]. 


PROoF. Step 0: If f has no fixed point in J, then by Theorem 10.40 either 
sup I or inf J is an attracting point for f. Thus we may assume that f has a fixed 
point L € I, and our task is to show that L is attracting for f. 

Step 1: Let x € J, and for n € N, put d, = |x, — L|. If for some N we have 
zy = L, then x, = LF for all n > N, so the sequence of iterates converges to L. 
So we may assume d, > 0 for all n € N, and then by weak contractivity {d,,} is 
decreasing. Since 0 is a lower bound, there is d > 0 such that d, — d. Observe 
that the desired conclusion that x, — LD is equivalent to d = 0. 
Step 2: For alln € N, a, € [L —do,L 4+ do], so {a} is bounded; by Bolzano- 
Weierstrass, there is a convergent subsequence, say tn, — y. As k > oo we have: 
[Zng =| == L, Zn, mae = dn, > d, 

|f(@n,) — LE] = |angti — L] = dnti > d, |f(an,) — L] > |f(y) — ZI, 
so |y — L| =d=|f(y) — L| = |f(y) — f(L)|. By weak contractivity, d = 0. 
Remark: The case J = R of Theorem 10.41 is due to A. Beardon [Be06]. The case 
I = [a, }] is an instance of a general result of M. Edelstein [Ed62] which is described 


in the next section. Our proof of Theorem 10.41 draws ideas from Beardon’s proof 
and also from K. Conrad’s treatment of Edelstein’s Theorem in [CdC]. 


Exercise: Show that the function f : [0,1] — [0,1] defined by f(x) = es is 
weakly contractive but not contractive. 


10.5. Fixed Point Theorems in Metric Spaces. 


Theorem 10.39 is a special case of a celebrated theorem of Banach,® itself called 
either the Contraction Mapping Principle or Banach’s Fixed Point Theo- 
rem. The flavor of the generalization can perhaps be appreciated by looking back 
at what tools we used to prove Theorem 10.39. The general strategy of the proof 
was to show that any sequence of iterates of x under f was a Cauchy sequence. 
To show this, we used nothing other than the definition of a contractive mapping 
and (repeatedly) the triangle inequality. 

Now there is a general mathematical structure called a metric space: this is 
a set X equipped with a metric function d: X x X > R. The intuition is that 
for x,y € X, d(x,y) measures the distance between x and y. When X = R, this 
function is nothing else than d(z,y) = |x — y|. In order for a function d to be a 
metric, it must satisfy three very familiar properties: 


(M1) (Definiteness) For all z,y € X, d(x, y) > 0, with equality iff « = y. 


Stefan Banach (1892-1945) 
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(M2) (Symmetry) For all x,y € X, d(x, y) = d(y,2). 
(M3) (Triangle Inequality) For all 2, y,z € X, d(x, z) < d(x, y) + d(y, z). 


Thus d(x, y) = |x — y| is certainly a metric on R. It turns out that these three 
simple properties enable us to carry over a large portion of the theory of real in- 
finite sequences to study sequences with values in a metric space X, i.e., functions 
ze: Zt — X. Essentially, wherever in any of our definitions we see |x — y|, we 
replace it with d(x, y). For instance, we say that a sequence {z,,} in a metric space 
X converges to an element L € X if for all « > 0, there is N € Z* such that for 
alln > N, d(ap,L) < €. Similarly, one can directly carry over the definition of a 
Cauchy sequence: try it! 


Similarly, if (X,dx) and (Y,dy) are metric spaces, then a function f : X > Y 
is continuous at x € X if for all « > 0, there exists 6 > 0 such that for all 2’ © X 
with dx(a, 2’) < 6, dy(f(x), f(a’)) < «. A function f is continuous if it is contin- 
uous at every point of X. (The reader should recognize this as a rather straight- 
forward adaptation of our cherished e-6 definition of continuity to this new context.) 


One important class of examples of metric spaces are obtained simply by taking a 
subset X C R and definining d(x,y) : X x X — Ras before: d(z,y) = |x — y|. 
The point here is that for some subspaces of X, a Cauchy sequence with values 
in X need not converge to an element of X. This is not really new or surprising: 
for instance, consider the sequence 2, = 4 as a sequence in (0,00): it is a Cauchy 
sequence there, but it does not converge to an element of (0, 00); rather it converges 
to 0. Similarly, in a general metric space X a Cauchy sequence need not converge 
to an element of X: we say that a metric space X is complete if every Cauchy 
sequence in X is convergent in X. Now we can state the theorem. 


THEOREM 10.42. (Banach Fixed Point Theorem [Ba22]) Let X be a complete 
metric space, and let f : X — X be a contraction mapping: that is, there is 
C € (0,1) such that for all x,y € X, 


d(f(x), f(y)) < Cd(z, y). 


Then: 

a) There is a unique L € X with f(L) = L, i.e., a fixed point of f. 

b) For every x9 € X, define the sequence of iterates of xo under f by tn41 = f(Xn) 
forn EN. Then for alln EN, 


d(x,,L) < C"d(xo, L). 


The proof of Theorem 10.42 is exactly the same as that of Theorem 10.39. 


For the last century, Banach’s Fixed Point Theorem has been one of the most 
important and useful results in mathematical analysis: it gives a very general con- 
dition for the existence of fixed points, and a remarkable number of “existence 
theorems” can be reduced to the existence of a fixed point of some function on 
some metric space. For instance, if you continue on in your study of mathematics 
you will surely learn about systems of differential equations, and the most impor- 
tant result in this area is that — with suitable hypotheses and precisions, of course 
— every system of differential equations has a unique solution. The now standard 
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proof of this seminal result uses Banach’s Fixed Point Theorem!’ 


Let X be a metric space. Then the statement “Every sequence with values in 
X admits a convergent subsequence” is certainly meaningful, but — as is already 
the case with intervals on the real line! — whether it is true or false certainly de- 
pends on X. We say that a metric space is compact if every sequence with values 
in X admits a convergent subsequence. 


In fact every compact metric space is complete, and the proof again requires no 
ideas other than the ones we have already developed: indeed, if {x,} is a Cauchy 
sequence in a compact metric space, then by definition of compactness it admits 
a subsequence 2, converging to some L € X, and then we prove — exactly as we 
did before — that if a subsequence of a Cauchy sequence converges to LZ then the 
Cauchy sequence itself must converge to L. 


Above we showed that a weakly contractive map on a closed, bounded (and thus 
compact) interval was attractive and attributed this to M. Edelstein. What Edel- 
stein actually showed was the following result. 


THEOREM 10.48. (Edelstein [Ed62]) Let X be a compact metric space, and let 
f:X > X be a weakly contractive mapping. Then f is attractive: 
a) There is a unique fixed point L of f. 
b) For all xp € X, the sequence of iterates of xo under f converges to f. 


PRoor. We follow [CdC]. 
Step 0: The Extreme Value Theorem has the following generalization to compact 
metric spaces: if X is a compact metric space, then any continuous function f : 
X — R is bounded and attains its maximum and minimum values. Recall that 
we gave two proofs of the Extreme Value Theorem for X = [a,}]: one using Real 
Induction and one using the fact that every sequence in [a,b] admits a convergent 
subsequence. Since by definition this latter property holds in a compact metric 
space, it is the second proof that we wish to carry over here, and we ask the 
interested reader to check that it does carry over with no new difficulties. 

Step 1: We claim f has a fixed point. Here we need a new argument: the one we 
gave for X = [a, 6] used the Intermediate Value Theorem, which is not available in 
our present context. So here goes: let g: X — R by g(x) = d(a, f(x)). Since f is 
continuous, so is g and thus the Extreme Value Theorem applies and in particular 
g attains a minimum value: there is L € X such that for all y € X, d(L, f(L)) < 
d(y, f(y)). But if f(L) # L, then by weak contractivity we have d(f(L), f(f(L))) < 
d(L, f(L)), ie., g(f(L)) < g(L), contradiction. So L is a fixed point for f. 

Step 2: The argument of Step 2 of the proof of Theorem 10.41 carries over directly 
to show that L is an attracting point for f. 


11. Extending Continuous Functions 


Let J C R be an interval.® A subset S of J is dense in I if for all x € J and all 
€ > 0, there exists at least one element of s with s € (w —e€,4+ €). 


TIn fact the title of [Ba22] indicates that applications to integral equations are being explicitly 
considered. An “integral equation” is very similar in spirit to a differential equation: it is an 
equating relating an unknown function to its integral(s). 

8To evade trivialities, we assume that all of our intervals contain more than one point. 
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For example, for every interval J the subset JM Q of rational numbers in J is 
dense in I, as is the subset JM (R \ Q) of irrational numbers in I. 


Here we wish to consider the following problem: suppose we are given a dense 
subset S' of J and a function f : S + R. Under which circumstances does f ex- 
tend to a function f :IT—>R? 

Stated this way the answer is easy: always. For instance, we may set 7 (x) =0 
for all x € I \ S. So we were not careful enough. Here’s what we really mean: 


QUESTION 10.44. Let S be a dense subset of the interval I, and suppose we 
are given a function f:S—>R. Under what circumstances is there a continuous 
function f : I > R extending f, t.e., such that f(x) = f(x) for alla eS? 


Here are some preliminary observations. First, the uniqueness problem is easier to 
solve than the existence problem, and it is helpful to nail that down first. 


PROPOSITION 10.45. (Uniqueness of Extension) Let S be a dense subset of an 
interval I, and let f : S — R be a function. There is at most one continuus 
extension of f to I: that is, if fist : I > R are two continuous functions such 
that fi(x) = fo(a) = f(a) for alla € S, then f,(x) = fo(x) for all x € I. 

PROOF. Step 1: First suppose f = 0 on S. Since S is dense in J, for any x € I 
and n € Zt, there is z, € S with |x, — 2| < 4. Thus we get a sequence {x,}°, 
taking values in S such that 7, — x. Let f : I > R be any continuous extension 
of f. Then since t, > 2, f(t) > f(x). But {f(vn)} = 0,0,0,..., so it converges 
(uniquely!) to 0 and thus f(x) = 0. 

Step 2: In general, suppose Fi; ie : IT + R are two continuous extensions of f, 
and put g = fi — fo: I + R. Then @ is continuous and for all x € S, g(x) = 
fila) — fo(x) = 0. By Step 1, for all x € J, 


0 = G(x) = file) — fale), 
so fi(x) = fo(2) for all x € I. 


To go further, we need the definition of a continuous function f : S — R for an 
arbitrary subset S of R. There is nothing novel here: f is continuous at so € S if for 
all € > 0, there is 6 > 0 such that for all s € S with |s — so| < 6, | f(s) — f(s0)| <«¢, 
f : S > R is continuous if it is continuous at s for every s € S. 


PROPOSITION 10.46. If f admits a continuous extension a to I, then f is 
continuous on S.. 


PROOF. Let so € S, and fix e > 0. Since S C if is continuous at so, so there 
is 6 > 0 such that for all x € I with |x — so| < 6, | f(x) — f(so0)| < €. In particular 
then, for all s € S with |s — so| < 6, | f(s) — f(so)| = | f(s) — f(so)| <«. 


Example: I = R, S =R\ {V2}, f(z) =0 for x < V2, f(x) =1 for > V2. Then 


f is continuous on S' but admits no continuous extension to I. 


Here is a key observation: our counterexample function f is continuous on S but 
not uniformly continuous there: there is no 6 > 0 such that for all 71,272 € S, 
|z1 — x2] <6 => |f(a1) — f(a2)| <1. 
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THEOREM 10.47. (Extension Theorem) Let S be a dense subset of an interval 
I, and let f : S > R be a function. If f is uniformly continuous on S', then it 
extends uniquely to a uniformly continuous function f :I > R. 


PROOF. As above, since S is dense in J, for each x € I there is a sequence 
{an} in § with a, + x. This suggests that we define f(x) = limy+oo f(an). And 
we will, but there are several things we need to check: 

(i) The sequence f(a,,) is convergent. 

(ii) For any other sequence {b, } in S with bp > 2, limn—+sco f(an) = limn—+oo f (bn). 
(iii) The (now well-defined) extension function f is continuous on J. 

So let’s do it: 

(i) By Theorem 10.32, it suffices to show that {f(a,)} is Cauchy. Fix « > 0. 
Since f is uniformly continuous on S, there exists 6 > 0 such that |s; — s2| < 
5 => |f(si) — f(s2)| < €. Since a, > 2, there is N € Z* such that forn > N, 
lan, —2| < 3. Then for m,n > N, |a@m — an| <5, 80 |f(@m) — f(an)| <e. 

(ii) Suppose that a, — x and b, — x. By (i), we know that f(a,) — L and 
f (bn) > M, say, and we must show that L = M. Fix « > 0. By uniform continuity 
there exists 6 > 0 such that |s; — s2| <6 ==> |f(si) — f(s2)| < €. Moreover for all 
sufficiently large n we have |a, — 2| < 2, |bn — x| < $, 80 |an — bp | < 6 and hence 
|f(an) — f(bn)| < e. It follows that |L — M| < e. Since € was arbitrary, L = M. 
(iii) Let 2 € I, and suppose that f is not uniformly continuous on J: then there 
exists some € > 0 such that for all 6 > 0, there are 71,22 € I with |x, — x2| < $ 
but |f(v1) — f(x2)| > €. By (ii), there are 51,52 € S such that 


ls1 — ail, |2—2al < 3, |F(s1) — f(ea)|,F(82) — F(@2)I < 5 


Then 


|s1 — s2| < |s1 — 21| + |x — 22] + [x2 — 82| <0, 


and 


|f(a1)—F(@2)| < |F(@1)—F (si) I+ 1F (81) -F(s2) I+ 1F (82) F(a) < * +1f(s1)-F(02)h 


But since f : S > R is uniformly continuous, we may take 6 to be sufficiently small 
so that |s1 — so] <6 ==> |f(si) — f(s2)| < §. For such a 6 we get 


2e € 


fle) -— flaal< 3 +3 =6 


contradiction. 


Theorem 10.47 gives necessary and sufficient conditions for a function f : S > R to 
admit a uniformly continuous extension to J. When I is closed and bounded, this 
solves our extension problem because uniform continuity is equivalent to continuity. 
However, for an interval which is not closed and bounded, being uniformly contin- 
uous is much stronger than being continuous. For instance, the only polynomial 
functions which are uniformly continuous on all of R are the linear polynomials. 

However there is an easy, but sneaky, way to “soup up” the Extension Theo- 
rem: if J is not closed and bounded, we don’t have to extend f to J “all at once”; 
we can do it via an increasing sequence of closed, bounded subintervals of J. 
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COROLLARY 10.48. Let S be a dense subset of R, and let f: S > R. The 
following are equivalent: 
(i) For all M > 0, the restriction of f to SN [—M, M] is uniformly continuous. 
(it) There is a unique extension of f to a continuous function ci :ROR. 


PRoor. (i) => (ii): Applying the Extension Theorem to f on SN [—M, M], 
we get a unique continuous extension fu : [-M,M] > R. Since the extensions 
are unique, for any x € R, we may choose any M with M > |a| and define f (t) = 
ti m(a): this does not depend on which M we choose. Moreover, since continuity at 
a point depends only on the behavior of the function in small intervals around the 
point, it is immediate that any function constructed from an expanding family of 
continuous functions in this way is continuous on all of R. 


CHAPTER 11 


Infinite Series 


1. Introduction 


1.1. Zeno Comes Alive: a historico-philosophical introduction. 


Humankind has had a fascination with, but also a suspicion of, infinite processes 
for well over two thousand years. Historically, the first kind of infinite process that 
received detailed infomation was the idea of adding together infinitely many quan- 
titties; or, to put a slightly different emphasis on the same idea, to divide a whole 
into infinitely many parts. 


The idea that any sort of infinite process can lead to a finite answer has been 
deeply unsettling to philosophers going back at least to Zeno,' who believed that a 
convergent infinite process was absurd. Since he had a sufficiently penetrating eye 
to see convergent infinite processes all around him, he ended up at the lively con- 
clusion that many everyday phenomena are in fact absurd (so, in his view, illusory). 


We will get the flavor of his ideas by considering just one paradox, Zeno’s ar- 
row paradox. Suppose that an arrow is fired at a target one stadium away. Can 
the arrow possibly hit the target? Call this event E. Before EF can take place, 
the arrow must arrive at the halfway point to its target: call this event E,. But 
before it does that it must arrive halfway to the halfway point: call this event FE. 
We may continue in this way, getting infinitely many events F,, Eo,... all of which 
must happen before the event &. That infinitely many things can happen before 
some predetermined thing Zeno regarded as absurd, and he concluded that the ar- 
row never hits its target. Similarly he deduced that all motion is impossible.” 


Nowadays we have the mathematical tools to retain Zeno’s basic insight (that a 
single interval of finite length can be divided into infinitely many subintervals) 
without regarding it as distressing or paradoxical. Indeed, assuming the arrow 
takes one second to hit its target and (rather unrealistically) travels at uniform 
velocity, we know exactly when these events FE; take place: E; takes place after 
4 seconds, E> takes place after + seconds, and so forth: EF, takes place after x 
seconds. Nevertheless there is something interesting here: we have divided the total 
time of the trip into infinitely many parts, and the conclusion seems to be that 


i wi i 
(80) Pel cod t..=1. 


1Zeno of Elea, ca. 490 BC - ca. 430 BC. 
2One has to wonder whether he got out much. 
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So now we have not a problem not in the philosophical sense but in the mathematical 
one: what meaning can be given to the left hand side of (80)? Certainly we ought 
to proceed with some caution in our desire to add infinitely many things together 
and get a finite number: the expression 


Lila: 


represents an infinite sequence of events, each lasting one second. Surely the ag- 
gregate of these events takes forever. 


We see then that we dearly need a mathematical definition of an infinite series 
of numbers and also of its sum. Precisely, if a1, a2,... is a sequence of real numbers 
and S is a real number, we need to give a precise meaning to the equation 


a, +...t¢@n+...= 58. 


So here it is. We do not try to add everything together all at once. Instead, we 
form from our sequence {a,,} an auxiliary sequence {5;,} whose terms represent 
adding up the first n numbers. Precisely, for n € Z*, we define 


Sn =a, +...+ ap. 


The associated sequence {S,,} is said to be the sequence of partial sums of the 
sequence {a,}; when necessary we call {a,,} the sequence of terms. Finally, we 
say that the infinite series a, +...+a,+...= Doeer Gdn converges to S — or 
has sum § - if lim, .. $, = S' in the familiar sense of limits of seqeunces. If 
the sequence of partial sums {S,,} converges to some number S' we say the infinite 
series is convergent; if the sequence {S,,} diverges then the infinite series }7°°_, an 
is divergent. 

Thus the trick of defining the infinite sum Sean Gn is to do everything in terms 
of the associated sequence of partial sums S, = a1 +...+ dn. 

In particular by $°°°_, a, = 00 we mean the sequence of partial sums diverges 
to co, and by 7°, an = —oo we mean the sequence of partial sums diverges to 
—oo. So to spell out the first definition completely, ae Gn = CO means: for every 
M ER there exists N € Zt such that for alln > N, a, +... +a, >M. 


Let us revisit the examples above using the formal definition of convergence. 


Example 1: Consider the infinite series 1+1+...+1+..., in which a, = 1 
for alln. Then S, =a, +...+a@, =1+...+1= 7, and we conclude 


1= lim n=o. 


noo 
n=1 


Thus this infinite series indeed diverges to infinity. 


Example 2: Consider t+ + ee x +..., in which a, = + for all n, so 
1 1 

81 S,=at...¢—. 

(81) eae 


There is a standard trick for evaluating such finite sums. Namely, multiplying (81) 
by 4 and subtracting it from (81) all but the first and last terms cancel, and we get 
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1 1 1 1 
Sn = Sn— 55m = 5-5 
and thus i 
Sn =1-—. 
Qn 
It follows that zs 
<= ]i 1-—)=1 
Do an = lim. — 5a) 


So Zeno was right! 


Remark: It is not necessary for the sequence of terms {a,} of an infinite series 
to start with a,. In our applications it will be almost as common to consider series 
starting with ap. More generally, if N is any integer, then by paw N Gn We mean 
the sequence of partial sums ay ,an +@Nn+41, Gn +4N41 + 4N42,.--- 


1.2. Telescoping Series. 


Example: Consider the series >, Tan We have 


1 
Si = 5, 

Lt 2 
S2=S5ita=5 6 3? 
95 = 92-405 = 5 BAT 
Si = 55 +04 = mF 


+ - 2 90 = NS igs + ta Te 
It certainly seems as though we have S,, = 1 — Gat ad for alln € Z. If this is 


the case, then we have 


- n 
a Qn = lim —l 
n>on+tl 
n=1 
How to prove it? 
First Proof: As ever, induction is a powerful tool to prove that an identity holds 
for all positive integers, even if we don’t really understand why the identity should 


hold! Indeed, we don’t even have to fully wake up to give an induction proof: we 
wish to show that for all positive integers n, 


” 1 n 
82 Sn = a ; 
(82) Bak n+1 
When n = 1, both sides equal s. Now suppose (82) holds for some n € Zt. Then 
S; =8$ t ! in M { l = Me | 1 
men en (atl? +(n4l) ntl n2+38n¢20 ntl (n+1(n+2) 
re a a 


(n+1)(n+2)  (n+1)(n+2) n+2° 


This approach will work whenever we have some reason to look for and successfully 
guess a simple closed form identity for S,,. But in fact, as we will see in the coming 
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sections, in practice it is exceedingly rare that we are able to express the partial 
sums S,, in a simple closed form. Trying to do this for each given series would turn 
out to be a discouraging waste of time. We need some insight into why the series 
poe win happens to work out so nicely. 

Well, if we stare at the induction proof long enough we will eventually notice how 
convenient it was that the denominator of aay factors into (n+ 1)(n + 2). 


Equivalently, we may look at the factorization Does this remind 


= 1 
n?4+n ~— (n+1)(n)* 
us of anything? I certainly hope so: this is a partial fractions decomposition. In 
this case, we know there are constants A and B such that 

1 aso 


nin+1) on’ n+l’ 


I leave it to you to confirm — in whatever manner seems best to you — that we have 
1 1 1 


n(nt1) on n+l 


This makes the behavior of the partial sums much more clear! Indeed we have 


1 
S,;=1- 5° 
1 ine 1 
So = S,+a,=(1 =1 : 
2= S51 +a2= (1-5) + (5-3) 3 
1 1 1 1 
S3 = So +a3 = (1 gy he pat 4’ 
and so on. This much simplifies the inductive proof that S, = 1 — an In fact 
induction is not needed: we have that 
1 1. ok 1 1 1 
Tine Gale oy be a) ares n+ 
the point being that every term except the first and last is cancelled out by some 
other term. Thus once again pyar aa limy+o 1 - ait = 1; 


Finite sums which cancel in this way are often called telescoping sums, I be- 
lieve after those old-timey collapsible nautical telescopes. In general an infinite 
sum )*~~_, a, is telescoping when we can find an auxiliary sequence {b,,}°2, such 
that a, = b; and for all n > 2, ay, = by — by_1, for then for all n > 1 we have 


Sn =a, +aQ+...+ an = bi + (bg bi) +...+ (On bn—1) = bn. 


But looking at these formulas shows something curious: every infinite series is 
telescoping: we need only take b, = S, for all n! Another, less confusing, way to 
say this is that if we start with any infinite sequence {S,, }°°,, then there is a unique 
sequence {an }?2, such that S, is the sequence of partial sums S;, = a1 +...+4n. 
Indeed, the key equations here are simply 


Si = a1, 
vn > 2, Sn = Sn-1 = an, 
which tells us how to define the a,,’s in terms of the S',’s. 


In practice all this seems to amount to the following: if you can find a simple 
closed form expression for the nth partial sum S,, (in which case you are very 
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lucky), then in order to prove it you do not need to do anything so fancy as mathe- 
matical induction (or fancier!). Rather, it will suffice to just compute that S; = a1 
and for all n > 2, S, — S,_1 = ay. This is the discrete analogue of the fact that if 
you want to show that f fdxz = F — i.e., you already have a function F which you 
believe is an antiderivative of f — then you need not use any integration techniques 
whatsoever but may simply check that F’ = f. 


Exercise: Let n € Z*. We define the nth harmonic number H,, = )>;_; ¢ = 
++ 4 +...+ 1. Show that for alln > 2, H, € Q\Z. (Suggestion: more specifically, 
show that for all n > 2, when written as a fraction ¢ in lowest terms, then the 
denominator b is divisible by 2.)° 


Exercise: Let k € Z*. Use the method of telescoping sums to give an exact formula 
for Sab in terms of the harmonic number Hy of the previous exercise. 

2. Basic Operations on Series 
Given an infinite series )*°°_, a, there are two basic questions to ask: 


QUESTION 11.1. For an infinite series Sy, an: 
a) Is the series convergent or divergent? 
b) If the series is convergent, what is its sum? 


It may seem that this is only “one and a half questions” because if the series diverges 
we cannot ask about its sum (other than to ask whether it diverges to too or “due 
to oscillation” ). However, later on we will revisit this missing “half a question”: if 
a series diveges we may ask how rapidly it diverges, or in more sophisticated lan- 
guage we may ask for an asymptotic estimate for the sequence of partial sums 


ae Gy, as a function of N as N > oo. 


Note that we have just seen an instance in which we asked and answered both 
of these questions: for a geometric series )>°°_, cr", we know that the series con- 


verges iff |r| < 1 and in that case its sum is —. We should keep this success 


story in mind, both because geometric series are ubiquitous and turn out to play 
a distinguished role in the theory in many ways, but also because other examples 
of series in which we can answer Question 11.1b) — i.e., determine the sum of a 
convergent series — are much harder to come by. Frankly, in a standard course 
on infinite series one all but forgets about Question 11.1b) and the game becomes 
simply to decide whether a given series is convergent or not. In these notes we try 
to give a little more attention to the second question in some of the optional sections. 


In any case, there is a certain philosophy at work when one is, for the moment, 
interested in determining the convergence / divergence of a given series )7°~_, an 
rather than the sum. Namely, there are certain operations that one can perform 
on an infinite series that will preserve the convergence / divergence of the series — 
i.e., when applied to a convergent series yields another convergent series and when 
applied to a divergent series yields another divergent series — but will in general 


3This is a number theory exercise which has, so far as I know, nothing to do with infinite 
series. But I am a number theorist... 
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change the sum. 


First, we may add or remove any finite number of terms from an infinite series 
without affecting its convergence. In other words, suppose we start with a se- 
ries )07-; dn. Then, for any integer N > 1, consider the series )77° yj) @n = 
Gn41+@Gn42+.... Then the first series converges iff the second series converges. 
Here is one (among many) ways to show this formally: write S,, = a; +...+ Gn 
and T, = an41+4ni2+...+4Nn4n- Then for all n € Zt, 


N 
(5 on) p= oy booby baa onan = Sion 
k=1 


Thus if lim, +..T, = Saar Gy, exists, so does limy. Svyn = liMpso Sn = 

oe @ sely if °° ists, then so does li aa Tr = 
yp Gn. Conversely if $7, an exists, then so does limp—.co Sop) Ge + Tn = 
ope G + litn+4c0 Tn, hence lim +co Tn = op 41 Gn exists. 


Similarly, if we are so inclined (and we will be, on occasion), we could add finitely 
many terms to the series, or for that matter change finitely many terms of the 
series, without affecting the convergence. We record this as follows. 


PROPOSITION 11.2. The addition, removal or altering of any finite number of 
terms in an infinite series does not affect the convergence or divergence of the series 
(though of course it may change the sum of a convergent series). 


As the reader has probably already seen for herself, reading someone else’s formal 
proof of this result can be more tedious than enlightening, so we leave it to the 
reader to construct a proof that she finds satisfactory. 


Because the convergence or divergence of a series )>>°_, ay, is not affected by chang- 
ing the lower limit 1 to any other integer, we often employ a simplified notation 
>, @n When discussing series only up to convergence. 


PROPOSITION 11.3. Let \>°°_, Gn, 0°21 bn be two infinite series, and let a be 
any real number. 
a) If Sy an = A and Sy, bn = B are both convergent, then the series >, an+ 
by is also convergent, with sum A+ B. 
b) If 3°", dn = A is convergent, then so also is °°, adn, with sum aA. 


PRooF. a) Let An = a1+...+@n, Bn = bit...+bn and Cy, = ai+b1+...+ 
Qn + bn. Then A, — A and B, — B, so for any € > 0, there is N € Zt such that 
for alln > N, |An — A] < § and |B, — B| < §. It follows that for alln > N, 
€ 


Cn — (A+ B)| = |An + Bn — A- BI <|An- Al +|Bn - BIS S45 = 


b) Left to the reader. 


Exercise: Let }°,, an be an infinite series and a € R. 
a) If a=0, show that )*,, aa, = 0. 
b) Suppose a # 0. Show >7,, a, converges iff $*,, aan converges. 


Exercise: Prove the Three Series Principle: let 7, an, >, bn, Don Cn be three 
infinite series with c, = an + by for all n. If any two of the three series yen Qn; 
>, Ons Yon Cn Converge, then so does the third. 
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2.1. The Nth Term Test. 


THEOREM 11.4. (Nth Term Test) Let S>,, an be an infinite series. If 7, an 
converges, then an, — 0. 


PROOF. Let S$ = pa Gy. Then for all n > 2, an = Sp — Sy_ 1. Therefore 


lim a, = lim S, —S,_1 = lim S, — lim S,_,; =S—S=0. 
n—0o n—-0Co n—0o noo 


We will often apply the contrapositive form of Theorem 11.4: if for a series }>,, an 
we have a, ~ 0,* then >, dn diverges. 


Warning: The converse of Theorem 11.4 is not valid! It may well be the case 
that an — 0 but >>, a, diverges. Later we will see many examples. Still, when put 
under duress (e.g. while taking an exam) many students can will themselves into 
believing that the converse might be true. Don’t do it! 


ae be a rational function. The polynomial Q(z) has only finitely 


many roots, so we may choose N € Z* such that for all n > N, Q(x) 4 0. Show 


that if deg P > deg Q, then °° x, nt is divergent. 


2.2. The Cauchy Criterion. 


Exercise: Let 


Recall that we proved that a sequence {x,,} of real numbers is convergent if and 
only if it is Cauchy: that is, for all « > 0, there exists N € Z* such that for all 
m,n > N we have |r — %m| < €. 


Applying the Cauchy condition to the sequence of partial sums {S,, = a1+...+an} 
of an infinite series )~°°_, an, we get the following result. 


PROPOSITION 11.5. (Cauchy criterion for convergence of series) An infinite 
series )\~~ , dn converges if and only if: for every € > 0, there exists No € Zt such 


that for all N > No and all k EN, | ON *8 an| <e. 


Note that taking k = 0 in the Cauchy criterion, we recover the Nth Term Test for 
convergence (Theorem 11.4). It is important to compare these two results: the Nth 
Term Test gives a very weak necessary condition for the convergence of the series. 
In order to turn this into a necessary and sufficient condition we must require not 
only that a, — 0 but also a, + G@n41 — 0 and indeed that a, +...+@n4~ — 0 for 
a k which is allowed to be (in a certain precise sense) arbitrarily large. 

Let us call a sum of the form ppeaee =an +anyit...t+an4p a finite tail 
of the series >, an. As a matter of notation, if for a fixed N € Z* and allk EN 


we have | ake an| <, let us abbreviate this by 


co 
| SS an| <€. 
n=N 
N+k 


In other words the supremum of the absolute values of the finite tails | \7) "yy Gn 
is at most «. This gives a nice way of thinking about the Cauchy criterion. 


4This means: an > L # 0, or an diverges. 
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PROPOSITION 11.6. An infinite series \>>-_, an converges if and only if: for all 


€ > 0, there exists No € Zt such that for all N > No, | cry @n| <e. 


In other (less precise) words, an infinite series converges iff by removing sufficiently 
many of the initial terms, we can make what remains arbitrarily small. 


3. Series With Non-Negative Terms I: Comparison 
3.1. The sum is the supremum. 
Starting in this section we get down to business by restricting our attention to 
series a ay With a, > 0 for all n € Z*. This simplifies matters considerably 
and places an array of powerful tests at our disposal. 
Why? Assume a, > 0 for all n € Zt and consider the sequence of partial sums: 
Sy =a, <a, + a2 = $2 < ay + ag + a3 = $3, 


and so forth. In general, S41 — Sp, = Gni1 > 0, so the sequence of partial sums 
{S,,} is increasing. Applying the Monotone Sequence Lemma we get: 


PROPOSITION 11.7. Let 3°, Gn be an infinite series with a, > 0 for alln. Then 
the series converges if and only if the partial sums are bounded above, i.e., if and 
only if there exists M € R such that for alln, ay +...+@4n <M. Moroever if the 
series converges, its sum is precisely the least upper bound of the sequence of partial 
sums. If the partial sums are unbounded, the series diverges to oo. 


Because of this, when dealing with series with non-negative terms we may express 
convergence by writing )7,, d, < co and divergence by writing 57, dn = 00. 


3.2. The Comparison Test. 
Example: Consider the series >>, an Its sequence of partial sums is 


nar()+2 (rst (8): 


Unfortunately we do not (yet!) know a closed form expression for T,,, so it is 
not possible for us to compute lim, +... 7, directly. But if we just want to decide 


whether the series converges, we can compare it with the geometric series )>°—_, aan 
1 1 1 
Pa gg eer a, 
Since 4 <1 for all n € Z*, we have that for all n € Zt, or < x Summing these 


inequalities from k = 1 to n gives T,, < S, for all n. By our work with geometric 
series we know that S, <1 for all n and thus also T,, < 1 for all n. Therefore our 
given series has partial sums bounded above by 1, so 7°, she <1. In particular, 
the series converges. 


Example: conside the series S77, \/n. Again, a closed form expression for T, = 
V1+...+./n is not easy to come by. But we don’t need it: certainly T, > 
1+...+1= 7. Thus the sequence of partial sums is unbounded, so 7°", /n = 00. 
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THEOREM 11.8. (Comparison Test) Let >>”, Qn, 17-1 bn be two series with 
non-negative terms, and suppose that ay < by, for alln € Z*. Then 


CO [o<) 
Yn Shu 
n=1 n=1 
In particular: if \7,, bn < 00 then >), an < 00, and if 37, dn = co then D7, by = 00. 
ProorF. There is really nothing new to say here, but just to be sure: write 


Sn =a, t+...+4n, Ty, = by +... + dn. 
Since az < by for all k we have S, < T,, for all n and thus 


co co 
yon = sup S$, <supT, = Seg: 
n=1 t ig n=1 


The assertions about convergence and divergence follow immediately. 


3.3. The Delayed Comparison Test. 


The Comparison Test is beautifully simple when it works. It has two weaknesses: 
first, given a series }7,, dn we need to find some other series to compare it to. Thus 
the test will be more or less effective according to the size of our repertoire of known 
convergent /divergent series. (At the moment, we don’t know much, but that will 
soon change.) Second, the requirement that a, <b, for all n € Z* is rather incon- 
veniently strong. Happily, it can be weakened in several ways, resulting in minor 
variants of the Comparison Test with a much wider range of applicability. 


Example: Consider the series 


[oe) 


1 i el 1 1 
So = =14+14+54 be. .4 bee 
nl 27 263 O84 2-B-...n 


n=0 


We would like to show that the series converges by comparison, but what to compare 
it to? Well, there is always the geometric series! Observe that the sequence n! 
grows faster than any geometric r” in the sense that lim,_,~ a = oo. Taking 
reciprocals, it follows that for any 0 < r < 1 we will have 4 < 1 — not necessarily 
for all n € Z*, but at least for all sufficiently large n. For instance, one easily 
establishes by induction that A < sh if and only ifn > 4. Putting a, = J and 
by = + we cannot apply the Comparison Test because we have a, > b,, for all 
n >4 rather than for all n > 0. But this objection is more worthy of a bureaucrat 
than a mathematician: certainly the idea of the Comparison Test is applicable here: 


SEN. ot, ET edt. 8. 1 67 
Doge at eg 8 A ae 5 tg a 


So the series converges. More than that, we still retain a quantitative estimate on 
the sum: it is at most (in fact strictly less than, as a moment’s thought will show) 


a = 2.79166666.... (Perhaps this reminds you of e = 2.7182818284590452353602874714..., 


which also happens to be a bit less than ae It should! More on this later...) 


We record the technique of the preceding example as a theorem. 
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THEOREM 11.9. (Delayed Comparison Test) Let \°-°_,, 07-1 bn be two series 
with non-negative terms. Suppose that there exists N € Zt such that for alln > N, 
An <b,. Then 


oo N co 
So an < (do. -5,] + 30 bn. 
n=1 n=1 n=1 


In particular: if \7,, bn < 00 then >), an < 00, and if 37, dn = co then 7, by = 00. 


Exercise: Prove Theorem 11.9. 


Thus the Delayed Comparison Test assures us that we do not need a, < b, for 
all n but only for all sufficiently large n. A different issue occurs when we wish to 
apply the Comparison Test and the inequalities do not go our way. 


3.4. The Limit Comparison Test. 


THEOREM 11.10. (Limit Comparison Test) Let S>, an, >>, bn two series. Sup- 
pose that there exists N € Z*+ and M € R2° such that foralln >N,0<an < Mbp. 
Then if >>, bn converges, Y>,, dn converges. 


Exercise: Prove Theorem 11.10. 


COROLLARY 11.11. (Calculus Student’s Limit Comparison Test) Let S>,, an 
and \_,, bn be two series. Suppose that for all sufficiently large n both a, and by, 
are positive and limn_+o6 a = Le (0,0. 

a) If0 < L < ow, the series )>,, an and >>, bn converge or diverge together (i.e., 
either both converge or both diverge). 

b) If L= oo and 30, dn converges, then >>, bn converges. 

c) If L=0 and 9%, by converges, then )*,, an converges. 


PROOF. In all cases we deduce the result from the Limit Comparison Test. 
a) If 0 < L < ov, then there exists N € Z* such that 0 < Fbn < an < (2L)bn. 
Applying Theorem 11.10 to the second inequality, we get that if 7), bn Ne 
then 5°, @n converges. The first inequality is equivalent to 0 < b, < 2 an for all 
n > N, and applying Theorem 11.10 to this we get that if 3>,, an converges, then 
>=, bn converges. So the two series )7,, an, 5°, bn converge or diverge together. 
b) If L = ov, then there exists N € Z* such that for all n > N, an > by > 0. 
Applying Theorem 11.10 to this we get that if 7, converges, then 5, b,, converges. 
c) This case is left to the reader as an exercise. 


Exercise: Prove Corollary 11.11c). 

Example: We will show that for all p > 2, the p-series ey 2s zh converges. In 
pe it is enough os enon. this for P= =), ate for p > 2 we have for all n € Z* that 
n? <n? andthus 4 <4s80>>,4< For p = 2, we happen to know that 


ae 
oo 


1 1 1 
Benet i) =1 


and in particular that 5> 


5 Pe ‘ 
n Ein Wan IS close to +. More 


and b, 4 we have ay ~ bn, ie., 


SOE For large n 


precisely, putting a, = a 
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Applying Theorem 11.11, we find that >7,, En and >, + converge or diverge 


together. Since the former series converges, we deduce that )7,, ls converges, even 
though the Comparison Test does not directly apply. 
Exercise: Let i be a rational function such that the degree of the denominator 


minus the degree of the numerator is at least 2. Show 77-4, a converges. 


Exercise: Determine whether each of the following series converges or diverges: 
fore) sen = it 

a) )op=1 Sin sz. 
a 

b) So) =1 COS pa. 


3.5. Cauchy products I: non-negative terms. 
Let yo 4 Gn and $7°~ 4 by, be infinite series. Can we, in some sense, multiply them? 


In order to forestall possible confusion, let us point out that many students are 
tempted to consider the following “product” operation on series: 


Co co s Co 
(>: os) : (>: ) iss by Andn- 
n=0 n=0 n=0 

In other words, given two sequences of terms {a,}, {bn}, we form a new sequence 
of terms {a,,b,,} and then the associated series. In fact this is not a very useful 
candidate for the product. If )7,, a, = A and >>, bn = B, we want our “product 
series” to converge to AB. But for instance, take {a,} = {b,} = +4. Then 


an 
See Se 
n=0 n=0 1- 2 


so AB = 4, whereas 


- il 1 4 
d,antn ar i= 3° 


Unfortunately 4 #4, What went wrong? 


Plenty! We have ignored the laws of algebra for finite sums: e.g. 


(ao + a1 + a2) (bo + by + b2) = agbo +.a1b1 + agbz + agby + a1 bp + agbz + 4161 + aabo. 


The product is different and more complicated — and indeed, if all the terms are 
positive, strictly larger — than just aobo + a1b; + agbe. We have forgotten about 
the cross-terms which show up when we multiply one expression involving several 
terms by another expression involving several terms.” 


Let us try again at formally multiplying out a product of infinite series: 
(ao tai +...+an+...)(bo +b1 +... +bn +...) 


= agbo t aoby t a,bo { agbe + a,b, t agbo trade aobn { aybn—1 feaneet Anbo eae 


5To the readers who did not forget about the cross-terms: my apologies. But it is a common 
enough misconception that it had to be addressed. 
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The notation is getting complicated. In order to shoehorn the right hand side into 
a single infinite series, we need to either (i) choose some particular ordering of the 
terms a,b¢ or (ii) collect some terms together into an nth term. 


For the moment we choose the latter: we define for any n € N 


n 
ne s anbn—k _ aobn oe ayby—1 eee Anbn 
k=0 


and then we define the Cauchy product of }7**_, a, and )>~~ 4 bn to be the series 


yen -> 1eD ons) | 


k=0 


THEOREM 11.12. Let ign }am o> (on}PLo be two series with non-negative terms. 
Let Spee oan = A and pene oon = B. Putting c, = ee oAkOn—rk we have that 
So tn = AB. In portieulan, the Cauchy product series converges iff the two 
“factor series” Y\ an and Y>,, bn both converge. 


PROOF. We define another sequence, the box product, as follows: for N EN, 


N= DS ajb; = (ag +... +ay)(bo +... + bn) = An By. 
0<i,j<N 


Thus by the usual product rule for sequences, we have 


lim Oy = Jim An Bn = AB. 
00 


N- oo 


So the box product clearly converges to the product of the sums of the two series. 
This suggests that we compare the Cauchy product to the box product. The entries 
of the box product can be arranged to form a square, viz: 


N = Q0b9 + andi) +... + agbn 


taybp + a,b, +... +a1bn 


tanbo taNbi +...+taynbn. 
On the other hand, the terms of the Nth partial sum of the Cauchy product can 
naturally be arranged in a triangle: 


Cn = aobo 
+agb1 + abo 
+ agb2 + a,b; + agbo 


tagbs + a,bz + ab; + agbo 


+agby + a,bn_1 + aoabn_2 +... + anbo. 
Thus while Oy is a sum of (N + 1)? terms, Cy is asumof14+2+...4+N+1= 
W+)Gv+2) terms: those lying on our below the diagonal of the square. Thus in 
considerations involving the Cauchy product, the question is to what extent one 
can neglect the terms in the upper half of the square — i.e., those with ajb; with 
i+j>N-—as N gets large. 
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Here, since all the a;’s and 6;’s are non-negative and Uy contains all the terms 
of C'y and others as well, we certainly have 


Cn <Un = An By < AB. 
Thus C = limy 4. Cy < AB. For the converse, the key observation is that if we 
make the sides of the triangle twice as long, it will contain the box: that is, every 
term of Oy is of the form a,b; with 0 < 7,7 < N; thusi+j <2N so a,b; appears 
as a term in Coy. It follows that Coy > Up and thus 


= ii = ji > ii = ili = : 


Having shown both that C < AB and C > AB, we conclude 


4. Series With Non-Negative Terms II: Condensation and Integration 


We have recently been studying criteria for convergence of an infinite series }>,, ay 
which are valid under the assumption that a, > 0 for all n. In this section we place 
ourselves under more restrictive hypotheses: that for all n € N, dn41 > an > 0, 
i.e., that the sequence of terms is non-negative and decreasing. 


Remark: It’s no loss of generality to assume a, > 0 for all n. If not, ay = 0 for 
some N and, since the terms are assumed decreasing we have 0 = ay =an41=... 
and our infinite series reduces to the finite series soos Gy: this converges! 


4.1. The Harmonic Series. 


Consider }>°°_, +, the harmonic series. Does it converge? None of the tests 
we have developed so far are up to the job: especially, a, — 0 so the Nth Term 
Test is inconclusive. 

Let us take a computational approach by looking at various partial sums. Si 
is approximately 5.187. Is this close to a familiar real number? Not really. Next 
we compute Sis9 ¥ 5.591 and S299 ~ 5.878. Perhaps the partial sums never exceed 
6? (If so, the series would converge.) Let’s try a significantly larger partial sums: 
Sio00 & 7.485, so the above guess is incorrect. Since Sig59 * 7.584, we are getting 
the idea that whatever the series is doing, it’s doing it rather slowly, so let’s instead 
start stepping up the partial sums multiplicatively: 


Sio0 © 5.878. 


S93 © 7.4854. 

S94 & 9.788. 

Sio5 & 12.090. 
Now there is a pattern for the perceptive eye to see: the difference Sjp%+1 — Sox 
appears to be approaching 2.30... = log 10. This points to S,, ¥ logn. If this is so, 
then since logn — oo the series would diverge. I hope you notice that the relation 
between A and log n is one of a function and its antiderivative. We ask the reader 
to hold this thought until we discuss the integral test a bit late on. 
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For now, we give the following brilliant and elementary argument due to Cauchy. 


Consider the terms arranged as follows: 


ee oe ee ee 
Ce ah hao S gee eye ee 


ie., we group the terms in blocks of length 2". The power of 4 which begins each 
block is larger than every term in the preceding block, so if we replaced every term 
in the current block the the first term in the next block, we would only decrease 
the sum of the series. But this latter sum is much easier to deal with: 


Sa 1 1 1 1 fy, 819 od 
So-> r(o+i)+(S+ltst+i) t+... =l¢stiti. S00. 
n (5) € i) (; 8 8 =) 2°2°2 


n=1 


Therefore the harmonic series }>~~_, diverges. 


Exercise: Determine the convergence of )>,, —hr. 
n 


Exercise: Let a be a rational function with degQ — deg P = 1. Show that 


Saar on diverges.® 
We deduce the following result. 


PROPOSITION 11.13. For a rational function mee the series > an con- 
verges iff deg Q — deg P > 2. 


PrRooF. Left to the reader. 


4.2. The Condensation Test. 


The apparently ad hoc argument used to prove the divergence of the harmonic 
series can be adapted to give the following useful test, due to A.L. Cauchy. 


THEOREM 11.14. (Condensation Test) Let S>~_, an be an infinite series such 
that dn > Gn41 > 0 for alln EN. Then: 
a) We have 3, an < org 2 aan <2” Gn 
b) Thus the series )),, dn converges iff the condensed series )),, 2” agn converges. 


PROOF. We have 


lo e) 
Dad = 2+azg+a4+a5+a6+a7+agt+... 
foe) 
< ay { ag a2 t a4+a4+ a4 + O4 8ag t aa ) 2 agn 
n=0 
= (a1 +a2) + (a2 +44 +@4 + a4) + (a4 + ag + ag + Gg + ag +g + ag + ag) + (ag ) 
< (a1 +44) + (a2 + 2 +43 +43) + (a4 + G4 + G5 +45 +6 + a6 + a7 + a7) + (ag ) 


=2 3 Gy 
n=1 


6Take N larger than any of the roots of Q(x), so that every term in the sum is well defined. 
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This establishes part a), and part b) follows immediately. 


The Cauchy Condensation Test is, I think, an a priori interesting result: it says 
that, under the given hypotheses, in order to determine whether a series converges 
we need to know only a very sparse set of the terms of the series — whatever is 
happening in between aan and @gn+1 is immaterial, so long as the sequence remains 
decreasing. This is a very curious phenomenon, and of couse without the hypothesis 
that the terms are decreasing, nothing like this could hold. 

On the other hand, it may be less clear that the Condensation Test is of any 
practical use: after all, isn’t the condensed series }>,, 2"agn more complicated than 
the original series }7,, an? In fact the opposite is often the case: passing from the 
given series to the condensed series preserves the convergence or divergence but 
tends to exchange subtly convergent /divergent series for more obviously (or better: 
more rapidly) converging/diverging series. 


Example: Fix a real number p and consider the p-series’ oer +. Our task 


is to find all values of p for which the series converges. 


Step 1: The sequence a, = + has positive terms. The terms are decreasing iff 
the sequence n? is increasing iff p > 0. So we had better treat the cases p < 0 sepa- 
rately. First, if p < 0, then limy; +. a = lim,+o0 n!?! = 00, so the p-series diverges 
by the nth term test. Second, if p = 0 then our series is simply )~,, s+ =U, l= 00. 
So the p-series “obviously diverges” when p < 0. 


Step 2: Henceforth we assume p > 0, so that the hypotheses of Cauchy’s Conden- 
sation Test apply. We get that 57, n~? converges iff S>) 2"(2")-? = $0, 2?27™P = 
>, (2'-”)” converges. But the latter series is a geometric series with geometric 
ratio r = 2'~?, so it converges iff |r| < 1 iff 2?-! > 1 iff p > 1. 


Thus we have proved the following important result. 


THEOREM 11.15. For p ER, the p-series )., a converges iff p> 1. 


Example (p-series continued): Let p > 1. By applying part b) of Cauchy’s Con- 
densation Test we showed that >°°., + < oo. What about part a)? It gives an 
explicit upper bound on the sum of the series, namely 


SS n(9gn\—-p _ l-pyn _ 
Yi <Smeyr= veya ae. 
n=1 n=0 n=0 
For instance, taking p = 2 we get 
— 1 
Dara Siempre eate: 
n=1 


Using a computer algebra package I get 


1024 
1 
1< y —z = 1.643957981030164240100762569 .... 
nr 
n=1 


7Or sometimes: hyperharmonic series. 
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So it seems like }>*° | <> ~ 1.64, whereas the Condensation Test tells us that it 
is at most 2. (Note ties since the terms are positive, simply adding up any finite 


number of terms gives a lower bound.) 


The following holes gives a technique for using the Condensation Test to es- 
timate pie 1 + to arbitrary accuracy. 

Exercise: Let N be a non-negative integer. 

a) Show that under the hypotheses of the Condensation Test we have 


oo co 
; An < ; 2” dgntn. 
n=2N+4+1 n=0 


b) Apply part a) to show that for any p > 1, 


eas ee 
mp — 2Np (| —21-P)" 


n=2N+1 
5 foe) 1 — 1 ta ths . 4 Sie . 
Example: )77*_, Wioen: Ct = Zien 8 positive and decreasing (since its reciprocal 


is positive and increasing) so the Condensation Test applies. We get that the 
convergence of the series is equivalent to the convergence of 


2 1 1 
d 2” log 2” = ised =e 


so the series diverges. This is rather subtle: we know that for any € > 0, eee aa 
converges, since it is a p-series with p= 1+. But logn grows more slowly than 
n© for any € > 0, indeed slowly enough so that replacing n° with logn converts a 
convergent series to a divergent one. 


Exercise: Determine whether the series 5> converges. 


1 
n log(n!) 
Exercise: Let p,q,r be positive real numbers. 
a) Show that >> converges iff g > 1. 


b) Show that >7,, andes 
c) Find all values of p,q,r such that S> 


n nile n)4q 
converges iff p > 1 or (p=1 and q> 1). 

1 ‘ 
n aPlogn)i(loglognyr Converges. 


The pattern of Exercise X.X can be continued indefinitely, giving series which con- 
verge or diverge excruciatingly slowly, and showing that the difference between 
convergence and divergence can be arbitarily subtle. 


4.3. The Integral Test. 


THEOREM 11.16. (Integral Test) Let f : [1,c0) + R be a positive decreasing 
function, and forn € Z* put ayn = f(n). Then 


a < = f(a)dx < a 
n=2 u n=1 


Thus the series )>,, Gn converges iff the improper integral fae i x)dx converges. 
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PrRooF. This is a rare opportunity in analysis in which a picture supplies a 
perfectly rigorous proof. Namely, we divide the interval [1,00) into subintervals 
[n,n + 1] for all n € N and for any N € N we compare the integral ie f(a)dx with 
the upper and lower Riemann sums associated to the partition {1,2,...,N}. From 
the picture one sees immediately that — since f is decreasing — the lower sum is 
yo a, and the upper sum is Sale An, So that 

N41 


N N 
Das | F(a)dx < So an. 
n=2 1 n=1 


Taking limits as N — oo, the result follows. 


Remark: The Integral Test is due to Maclaurin® [Ma42] and later in more modern 
form to Cauchy [Ca89g]. 


Among series which arise naturally in undergraduate analysis, it usually holds that 
the Condensation Test can be successfully applied to determine convergence / di- 
vergence of a series if and only if the Integral Test can be successfully applied.® 


Example: Let us use the Integral Test to determine the set of p > 0 such that 


+ converges. Indeed the series converges iff the improper integral [~~ 22 is 
Done g g prop eral fy; $ 


P P 
° dx at? ie=e0 
ace pe aa 
ee a 


finite. If p ~ 1, then we have 
The upper limit is 0ifp—-1<0 = p> landis wifp<1. Finally, 
°° dx ee 
— =log2|S2, = ov. 
1 &£ 


So, once again, the p-series diverges iff p > 1. 


Exercise: Verify that all of the above examples involving the Condensation Test 
can also be done using the Integral Test. 


Given the similar applicability of the Condensation and Integral Tests, it is perhaps 
not so surprising that many texts content themselves to give one or the other. In 
calculus texts, one almost always finds the Integral Test, which is logical since often 
integration and then improper integation are covered earlier in the same course in 
which one studies infinite series. In elementary analysis courses one often develops 
sequences and series before the study of functions of a real variable, which is log- 
ical because a formal treatment of the Riemann integral is necessarily somewhat 
involved and technical. Thus many of these texts give the Condensation Test. 


The Condensation Test and the Integral Test have a similar range of applicabil- 
ity: in most “textbook” examples, where one test succeeeds, so will the other. 
From an aesthetic standpoint, the Condensation Test is more appealing (to me). 
On the other hand, under a mild additional hypothesis the Integral Test can be 
used to give asymptotic expansions for divergent series. Our treatment of the 


8Colin Maclaurin, 1698-1746 
2Why this should be so is not clear to me: the observation is purely empirical. 


246 11. INFINITE SERIES 
next two results owes a debt to [CdD]. 


Let us first establish some notation: Suppose that f,g : [a,co) > R and that 
g(x) 4 0 for all sufficiently large x. We write f ~ g and say f is asymptotic to 
f(t) _ 


g if limz50 Go 1. Similarly, given sequences {an}, {bn} with bn, #4 0 for all 


sufficiently large n, we write ay ~ by if limps a =1, 
Exercise: Let f and g be nonzero polynomial functions. Show that f ~ g iff f 
and g have the same degree and the same leading coefficient. 


LEMMA 11.17. Let {a,} and {b,} be two sequences of positive real numbers 
with an ~ bn and S>, dn = 00. Then Yo, bn = co and an An ~ Sey Dy. 


Proor. By the Limit Comparison Test, )7,, dn = 00. Now fix e > 0 and choose 
K €Z* such that for all n > K we have a, < (1+ €)bn. Then for N > K, 


N K-1 N K-1 N 
aS So an + SF dy So an + S> (1+ €)bn 
n=1 n=1 n=kKk n=1 n=Kk 


Ky K-1 N N 
= (Som (14 ot] t SO(1+€)bn = Cox + (1 +6) 5) bn, 


n=1 
say, where C.,~% does not depend on N. Dividing both sides by aaa b, and using 


N 
limpy— 300 peer bn = 00, we find that anal = is at most 1 + 2e for all sufficiently 


n=17°" 


large N. Because our hypotheses are symmetric in 5>,, a, and _,, bp, we also have 


that a is at most 1 + 2e for all suffiicently large N. It follows that 
n=1 En 


THEOREM 11.18. Let f : [1,00) > (0,00) be continuous and monotone. Sup- 
pose the series )_,, f(n) diverges and that as x — oo, f(x) ~ f(w+1). Then 


N N 
Syn) ~ | fla)de. 


PROOF. Case 1: Suppose f is increasing. Then, forn < x < n+1, we have 
f(n) < fP** fla)de < f(n +1), or 


fr" fla)de f(a 1), 


SFR) FR) 
By assumption we have 
wth SL) 
ea 


so by the Squeeze Principle we have 


n+l 
(83) ‘ fade ~ f(r). 
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Applying Lemma 11.17 with a, = f(n) and b, =f" 


f@ 
[teas E yf soon v= Ss) 


)dx, we conclude 


Further, we have 


JN" f@)de _ oo | f(N +1) 


=15 


where in the starred equality we have applied L’Hopital’s Rule and then the Fun- 
damental Theorem of Calculus. We conclude 


[tere [Hote ~ » i(n) 


Case 2: Suppose f is decreasing. Then for n < # <n+1, we have 


n+l 

finstys [ flojdr < F0), 

or 
n+l 
f(n+1) 2 fC flejde Ze 
f(r) ~  f(m) 

Once again, by our assumption that f(n) ~ f(n +1) and the Squeeze Principle we 
get (83); the remainder of the proof proceeds as in the previous case. 


5. Series With Non-Negative Terms III: Ratios and Roots 


We continue our analysis of series )7,, a, with a, > 0 for all n. In this section we 
introduce two important tests based on a very simple — yet powerful — idea: if for 
sufficiently large n a, is bounded above by a non-negative constant M times r” for 
0 <r <1, then the series converges by comparison to the convergent geometric 
series }>,, Mr”. Conversely, if for sufficiently large n a, is bounded below by a 
positive constant M times r” for r > 1, then the series diverges by comparison to 
the divergent geometric series >, Mr”. 


5.1. The Ratio Test. 


THEOREM 11.19. (Ratio Test) Let 3°, an be a series with ay > 0 for all n. 
a) Suppose there exists N € Z* and0 <r <1 such that for alln > N, —_ <r. 
Then the series S>, Gn converges. 
b) Suppose there exists N € Zt andr >1 such that for alln > N, ae >r. Then 
the series S>, Gn diverges. 
c) The hypothesis of part a) holds if p = limn+co 


An41 
@n 
d) The hypothesis of part b) holds if p = limn +. “** exists and is greater than 1. 


an 


exists and is less than 1. 


PROoF. a) Our assumption is that for all n > N, set <r< 1. Then 
Ant2 __ An+2 Anti 


* a <r’. An easy induction argument shows that for all k € N, 
n n+1 Qn 


QN+k 
an 


<r sa 
sO 
< k 
QN+k SGnNT . 
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Summing these inequalities gives 


co co co 
) = ; QN+k < ) anr® < oO, 


so the series 57, d, converges by comparison. 
b) Similarly, our assumption is that for alln > N, ee >r-> 1. As above, it 
follows that for all k € N, 
ON+k > rk, 
an 
so 
GN+k > ayr* >ayn >0. 
It follows that a, ~ 0, so the series diverges by the Nth Term Test. 
We leave the proofs of parts c) and d) as exercises. 


Exercise: Prove parts c) and d) of Theorem 11.19. 


x 


Example: Let « > 0. We will show that the series 77° , a converges. (Recall 
we showed this earlier when x = 1.) We consider the quantity 


grt 
Qnti — (mt # 
An ald n+1 


n 


= 0. Thus the series converges for any x > 0. 


It follows that lim,,-o6 auth 


5.2. The Root Test. 


In this section we give a variant of the Ratio Test. Instead of focusing on the 

property that the geometric series }>,, r” has constant ratios of consecutive terms, 

we observe that the nth root of the nth term is equal to r. Suppose now that 
1 


ae Gy is a series with non-negative terms such that af <r for some r < 1. Rais- 
ing both sides to the nth power gives an, < r”, and once again we find that the 
series converges by comparison to a geometric series. 

THEOREM 11.20. (Root Test) Let > an be a series with an > 0 for all n. 


a 
a) Suppose there exists N € Zt and0 <r <1 such that for alln > N, ap <r. 
Then the series S>, Gn converges. 


it 
b) Suppose that for infinitely many positive integers n we have a7 > 1. Then the 
series )_,, Gn diverges. 


a: 
c) The hypothesis of part a) holds if p = limn-+.0an exists and is less than 1. 

1 
d) The hypothesis of part b) holds if p = limp+. an exists and is greater than 1. 


Exercise: Prove Theorem 11.20. 


5.3. Ratios versus Roots. 
It is a fact — a piece of calculus folklore — that the Root Test is stronger than 
the Ratio Test. That is, whenever the ratio test succeeds in determining the con- 


vergence or divergence of a series, the root test will also succeed. 


In order to explain this result we need to make use of the limit infimum and limit 
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supremum. First we recast the ratio and root tests in those terms. 


Exercise: Let }°,, an be a series with positive terms. Put 


se 6 Ant1 : An+1 
p =liminf “*, p=limsup ©. 
~ neo An n—oco =6 an 


a) Show that if p < 1, the series >, an converges. 
b) Show that if p > 1 the series 7, an diverges. 


Exercise: Let 5°, @n be a series with non-negative terms. Put 
= 1 
? = limsupa7. 
n—->Co 
a) Show that if @ < 1, the series >, a, converges. 
b) Show that if > 1, the series )>,, an diverges.1° 


Exercise: Consider the following conditions on a real sequence {x,,}°2,: 
(i) limsup,,_,.5 @n > 1. 
(ii) For infinitely many n, @ > 1. 
(iii) lim sup,,_,,, Un > 1. 
a) Show that (¢) = > (di) => (di) and that neither implication can be reversed. 
b) Explain why the result of part b) of the previous Exercise is weaker than part 
b) of Theorem 11.20. 
1 


c) Give an example of a non-negative series >, an with 0 = limsup,_,,, af = 1 
such that 7, an = 00. 


PROPOSITION 11.21. For any series )),, Gn with positive terms, we have 


a Be ee i a 
p = liminf tid <@=liminfa, < @limsupa;” < p= limsup tore 


= n>0O An n—-0o noo n>co =n 


Exercise: Let A and B be real numbers with the following property: for any real 
number r, if A < r then B <r. Show that B < A. 


PROOF. Step 1: For any sequence {z,}, liminf a, < limsupzp, hence 6 < 6. 
Step 2: We show that 0 < p. For this, suppose r > /, so that for all sufficiently 
large n, ontt <r. As in the proof of the Ratio Test, we have ani, < r*a,, for all 
k € N. We may rewrite this as 

Ontk < ge (=) , 
or 


Peli a whe 
a+E n 
any kT (=) . 
Now 
1 
= 1 1 Gn \ wk 
6 =limsupan = limsupa,,, < limsupr (=) =r, 
noo k-00 k—-oo ee 
By the preceding exercise, we conclude 6 < . 
Step 3: We must show that p < 6. This is very similar to the argument of Step 2, 
and we leave it as an exercise. 


l0This is not a typo: we really mean the limsup both times, unlike in the previous exercise. 
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Exercise: Give the details of Step 3 in the proof of Proposition 11.21. 


Now let 5°, @n be a series which the Ratio Test succeeds in showing is conver- 
gent: that is, p < 1. Then by Proposition 11.21, we have @ < p < 1, so the Root 
Test also shows that the series is convergent. Now suppose that the Ratio Test 
succeeds in showing that the series is divergent: that is p > 1. Then? >@>p>1, 
so the Root Test also shows that the series is divergent. ~ 


Exercise: Consider the series )7,, 7 ead aa 
a) Show that p= $ and p = 2, so the Ratio Test fails. 
b) Show that 6 = @ = 4, so the Root Test shows that the series converges. 


Exercise: Construct further examples of series for which the Ratio Test fails but 
the Root Test succeeds to show either convergence or divergence. 


Warning: The sense in which the Root Test is stronger than the Ratio Test is 
a theoretical one. For a given relatively benign series, it may well be the case that 
the Ratio Test is easier to apply than the Root Test, even though in theory when- 
ever the Ratio Test works the Root Test must also work. 

Example: Consider again the series }>>~ 9 4. In the presence of factorials one 
should always attempt the Ratio Test first. Indeed 


! ! 
m 2"! — tim I/(n+1)! n! . 1 


n+cO On noo 1/n! — pares (n + 1)n! = pare n+l 


= 0. 


Thus the Ratio Test limit exists (no need for liminfs or limsups) and is equal to 0, 
so the series converges. If instead we tried the Root Test we would have to evaluate 
limnsoo(4)*. This is not so bad if we keep our head — e.g. one can show that for 


any fixed R > 0 and sufficiently large n, n! > R” and thus (4) < (<z)* ==: 

Thus the root test limit is at most z for any positive R, so it is 0. But this is 

elaborate compared to the Ratio Test computation, which was immediate. 
Turning these ideas around, Proposition 11.21 can be put to the following 


sneaky use. 


COROLLARY 11.22. Let {an}°2, be a sequence of positive real numbers. Assume 
a 


that limn,. = — L € [0,00]. Then also limn.. an = L. 


ProoF. Indeed, the hypothesis gives that for the infinite series }7,, dn we have 
p = L, so by Proposition 11.21 we must also have 6 = L. 


Exercise: Use Corollary 11.22 to evaluate the following limits: 
a) limp—yoo nz. 
b) For a ER, lim, 47 


c) limp. (n!) mn, 


a 
n 


6. Absolute Convergence 


6.1. Introduction to absolute convergence. 


We turn now to the serious study of series with both positive and negative terms. 
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It turns out that under one relatively mild additional hypothesis, virtually all of 
our work on series with non-negative terms can be usefully applied in this case. In 
this section we study this wonderful hypothesis: absolute convergence. (In the next 
section we get really serious by studying series in the absence of absolute conver- 
gence. This will lead to surprisingly delicate and intricate considerations. ) 


A real series }>, a, is absolutely convergent if 5°, |a,| converges. Note that 
>, |@n| is a series with non-negative terms, so to decide whether it is convergent 
we may use all the tools of the last three sections. A series }7,, dn, which converges 
but for which >>), |an| diverges is said to be nonabsolutely convergent.'! 


The terminology absolutely convergent suggests that the convergence of the se- 
ries )*,, |an| is somehow “better” than the convergence of the series }°,, dn. This is 
indeed the case, although it is not obvious. But the following result already clarifies 
matters a great deal. 


PROPOSITION 11.23. Every absolutely convergent real series is convergent. 


PROOF. We shall give two proofs of this important result. 

First Proof: Consider the three series )>,, an, >, |@n| and 5°, Gn + |an|. Our 
hypothesis is that 57, |an| converges. But we claim that this implies that 7, an + 
|a,| converges as well. Indeed, consider the expression a, + |a@,|: it is equal to 
2a, = 2|a,| when a, is non-negative and 0 when a,, is negative. In particular the 
series )>,, Gn + |ay| has non-negative terms and 37, an + |an| < 35, 2|Gn| < co. So 
>, an + |a@n| converges. By the Three Series Principle, )7,, a, converges. 

Second Proof: The above argument is clever — maybe too clever! Let’s try 
something more fundamental: since }>,, |a,| converges, for every € > 0 there exists 
N € Z* such that for all n > N, 3° y |an| < . Therefore for all k > 0, 


N+k N+k oo 


| So nl < S5 lanl < So lanl <¢, 
n=N n=N n=N 


and 5°, Gn converges by the Cauchy criterion. 


Exercise: If }>°° an is absolutely convergent, show that | 377° 4 @n| < 3729 lanl- 


Exercise: Find a sequence {a,,}°2, of rational numbers such that °°, |an| is 
; oo : be wd 
a rational number but )77~_, an is an irrational number. 


The main idea is that when we try to extend our rich array of convergence tests for 
non-negative series to series with both positive and negative terms, a sufficient (and 
often necessary) hypothesis is that the series be not just convergent but absolutely 
convergent. 


1lWe warn the reader that the more standard terminology is conditionally convergent. 
We will later on give a separate definition for “conditionally convergent” and then it will be a 
theorem that a real series is conditionally convergent if and only if it is nonabsolutely convergent. 
The reasoning for this — which we admit will seem abstruse at best to our target audience — is 
that in functional analysis one studies convergence and absolute convergence of series in a more 
general context, such that nonabsolute converge and conditional convergence may indeed differ. 
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Exercise: State and prove a Comparison Test and Limit Comparison Test for ab- 
solute convergence. 


THEOREM 11.24. (Ash [As12]) Let {a,}°2, and {b,}°2, be real sequences 
such that for alln € Z* we have b, #0. Suppose: 
(4) limn+00 = = 1, and 
(ii) 7°, an, diverges and )~°~_, by converges. 
Then: Y—, |bn| = 00 and limp+co Ga bn = 0. 


PROOF. If 3>°<, |bn| < co, then applying the limit comparison test to the abso- 
lute series )7°°., |an| and S7°., |bn|, we get that )°°°., an is absolutely convergent 
and thus convergent, contradicting (ii). Moreover 


Gn —b a 
lim ——* = lim “-1=0. 
Exercise: Show that if we weaken (i) to lim... |#| = L € (0,00), then still 


yO, On is not absolutely convergent. 
As an example of how Theorem 11.23 may be combined with the previous tests 


to give tests for absolute convergence, we record the following result. 


THEOREM 11.25. (Ratio & Root Tests for Absolute Convergence) Let 7, an 
be a real series. 
a) Assume a, #0 for all n. If there exists 0 <r <1 such that for all sufficiently 
large n, |“=**| <r, then the series 57, dn is absolutely convergent. 
b) Assume ay, #0 for all n. If there exists r > 1 such that for all sufficiently large 
n, |@+*|>r, the series )7,, Gn ts divergent. 


c) If there exists r < 1 such that for all sufficiently large n, |an|* <r, the series 
yan ts absolutely convergent. 


d) If there are infinitely many n for which lan] > 1, then the series diverges. 


ProorF. Parts a) and c) are immediate: applying Theorem 11.19 (resp. Theo- 
rem 11.20) we find that 5°, |an| is convergent — and the point is that by Theorem 
11.23, this implies that 7, dn is convergent. 

There is something to say in parts b) and d), because in general just because 
>, |@n| = co does not imply that 7, a, diverges. (We will study this subtlety 
later on in detail.) But recall that whenever the Ratio or Root tests establish the 
divergence of a non-negative series }7,, bn, they do so by showing that b, ~ 0. 
Thus under the hypotheses of parts b) and d) we have |a,,| ~ 0, hence also a, + 0 
so )°,, @n diverges by the Nth Term Test (Theorem 11.4). 


In particular, for a real series }>,, ay, define the following quantities: 


2 An+1 : : 
p= lim || when it exists, 
n—->co 
one Gn+1 
p = liminf |", 
a n—->co 
= 5 An+1 
p= limsup |“, 
n—->co an 


: 1 Doser! 
6 = lim |a,|” when it exists, 
n—-co 
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6 = limsup lan|*, 
n—-+>co 


and then all previous material on Ratio and Root Tests applies to all real series. 


THEOREM 11.26. Let 307° 9 an = A and > 9 bn = B be two absolutely con- 
vergent series, and let cn = peas Akbn_p. Then the Cauchy product series peeae (ony 
is absolutely convergent, with sum AB. 


PROOF. We have proved this result already when a,b, > 0 for all n. We wish, 
of course, to reduce to that case. As far as the convergence of the Cauchy product, 
this is completely straightforward: we have 


9°, co n oc on 
do lenl = SOLD aebnal < DIY laellbn—al < 00, 
n=0 n=0 k=0 n=0 k=0 

the last inequality following from the fact that peas peas |ax||bn—«| is the Cauchy 


product of the two non-negative series )>° 9 |an| and S°°~ 4 |b |, hence it converges. 


Therefore 57,, |cn| converges by comparison, so the Cauchy product series )7,, cn 
converges. 
We now wish to show that limy +6 Cn = pee Cn = AB. Recall the notation 


N= De aid; = (ao Peat an )(bo tessa ct bn) = AnBn. 
0<i,j<N 
We have 
|Cn — AB| < |On — AB| + |Ow — Cy 
_ |An By — AB| + jarby| + lagby-1| { |agbn| weit land | FS est lanbn| 
<|Aw By — AB| + (>: en! DY lenl } + (>. Pl) de lan 
n=0 n>N n=0 n>N 


Fix € > 0; since Ay By — AB, for sufficiently large N |AnBy — AB| < §. Put 


A=) lan|, B= >— |bnl. 
n=0 n=0 


By the Cauchy criterion, for sufficiently large N we have 7,,.y |bn| < 3, and 
Vendn [@n| < 3g and thus |Cy — AB <e. 


6.2. Cauchy products II: Mertens’s Theorem. 


While the proof of Theorem 11.26 may seem rather long, it is in fact a rather 
straightforward argument: one shows that the difference between the “box prod- 
uct” and the partial sums of the Cauchy product becomes negligible as N tends 
to infinity. In less space but with a bit more finesse, one can prove the following 
stronger result, a theorem of F. Mertens [Me72].'* 


THEOREM 11.27. (Mertens’ Theorem) Let 3° an = A be an absolutely con- 
vergent series and Smee bn = B be a convergent series. Then the Cauchy product 
series \ ~~ 9 Cn converges to AB. 


12Fyanz Carl Joseph Mertens, 1840-1927 
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Proor. (Rudin [R, Thm. 3.50]): define (as usual) 


N N N 
Ay = >) Gn, By => bas Cun= > en 
n=0 n=0 n=0 
and also (for the first time) 
Bn = By, — B. 
Then for all N EN, 
Cn = agbo t (agby t abo) I seat oh (aobn +... anbo) 


=aoBy +4a,Bny_1+...+anBo 
= ao(B+ By) +a1(B + By_-1) +...+an(B +t Bo) 


= AnB+ao6n +a18n-1+...+4n 60 = AnB+ yn, 


say, where yy = ao6y + a16n_-1+...+ nfo. Since our goal is to show that 
Cy — AB and we know that Ay B > AB, it suffices to show that yy > 0. Now, 
put a = 0, |an|. Since By > B, By — 0, and thus for any « > 0 we may choose 
No €N such that for all n > No we have |§,| < 3,. Put 


M= ; 
o<n=No In 


By the Cauchy criterion, for all sufficiently large N, M >7,3.v_n, |an| < €/2. Then 


lyn| < [Goan +... + Bn,aN—No| + [Ono +14N—No-1 +--+ Bnao| 


€ € € € 
< |Boan +... + Bvean—nol + 5 SM 3 lan| te Sgt oS e 


So yn > 0. 


7. Non-Absolute Convergence 


We say that a real series }>,, an is nonabsolutely convergent if the series con- 
verges but >, |an| diverges, thus if it is convergent but not absolutely convergent.'? 


A series which is nonabsolutely convergent is a more delicate creature than any 
we have studied thus far. A test which can show that a series is convergent but 
nonabsolutely convergent is necessarily subtler than those of the previous section. 
In fact the typical undergraduate student of calculus / analysis learns exactly one 
such test, which we give in the next section. 


13Qne therefore has to distinguish between the phrases “not absolutely convergent” and 
“nonabsolutely convergent”: the former allows the possibility that the series is divergent, whereas 
the latter does not. In fact our terminology here is not completely standard. We defend ourselves 
grammatically: “nonabsolutely” is an adverb, so it must modify “convergent”, i.e., it describes 
how the series converges. 
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7.1. The Alternating Series Test. 


Consider the alternating harmonic series 


n 2° 3 


n=1 


Upon taking the absolute value of every term we get the usual harmonic series, 
which diverges, so the alternating harmonic series is not absolutely convergent. 
However, some computations with partial sums suggests that the alternating har- 
monic series is convergent, with sum log 2. By looking more carefully at the partial 
sums, we can find a pattern that allows us to show that the series does indeed 
converge. (Whether it converges to log 2 is a different matter, of course, which we 
will revisit much later on.) 


It will be convenient to write a, = i, so that the alternating harmonic series 


i ntl . e . . 
is >, ( a . We draw the reader’s attention to three properties of this series: 


(AST1) The terms alternate in sign. 
(AST2) The nth term approaches 0. 
(AST3) The sequence of absolute values of the terms is weakly decreasing: 


Op 20S eS Oy Be 


These are the clues from which we will make our case for convergence. Here it is: 
consider the process of passing from the first partial sum S$; = 1 to S3 = 1— 4 +3 = 
3. We have $3 < 1, and this is no accident: since aj > a3, subtracting a2 and then 
adding a3 leaves us no larger than where we started. But indeed this argument is 


valid in passing from any S2n-1 to Son41: 
Sont1 = Son-1 — (@an — Gant) S Son-1- 


Thus the sequence of odd-numbered partial sums {52,1} is decreasing. Moreover, 


Sonti = (a1 — a2) + (a3 — a4) +... + (Gen—1| — |@an) + don—1 > 0. 


Therefore all the odd-numbered terms are bounded below by 0. By the Monotone 
Sequence Lemma, the sequence {.S2n41} converges to its greatest lower bound, say 
Soda. On the other hand, just the opposite sort of thing happens for the even- 
numbered partial sums: 


Son+2 = San + Gon+1 — Gan+2 = Son 


and 


Son+2 = ay, (a2 a3) (a4 as) tee (aan d2n+1)) — dan42 La. 


Therfore the sequence of even-numbered partial sums {.52,,} is increasing and 
bounded above by a1, so it converges to its least upper bound, say Seven. Thus we 
have split up our sequence of partial sums into two complementary subsequences 
and found that each of these series converges. Now the sequence {.5,,} converges iff 
Soda = Seven, and the inequalities 


Sy SiS in SS Si < Sa ale See Gh 
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show that Seven < Soaa- Moreover, for any n € Z* we have 


Soad — Seven < Sonti = Son = 42n+1- 


Since don41 > 0, we conclude Soaq = Seven = 5, i.e., the series converges. 


Further, since for all n we have Son < Sante < S < San41, it follows that 
|S — Sen| = S — Son < Sonti — Son = don+1 
and similarly 
IS — Songil = Santi — 9 < Santi — Sento = Gante2- 


Thus the error in cutting off the infinite sum S>°°_,(—1)"*"|a,,| after N terms is in 
absolute value at most the absolute value of the next term: ay41. 


Of course in all this we never used that a, = 4 but only that we had a series 
satisfying (AST 1) (i.e., an alternating series), (AST2) and (AST3). Therefore the 
preceding arguments have in fact proved the following more general result, due 
originally due to Leibniz. 


THEOREM 11.28. Let {a,}92, be a sequence of non-negative real numbers which 
is weakly decreasing and such that limyp+.o an = 0. Then: 
a) The associated alternating series >, (—1)"*t'a, converges. 
b) For N €Z*, put 


oo N 
(84) Ew =|()5(-1)"*4an) — ()o(-1)"@ 


Then we have the error estimate 


En < an41. 


_4)rt1 , 
( ) 1s: 


Exercise: Let p € R: Show that the alternating p-series )>>_, 
(i) divergent if p < 0, 

(ii) nonabsolutely convergent if 0 <p < 1, and 

(iii) absolutely convergent if p > 1. 


mP 


Exercise: Let ies be a rational function. Give necessary and sufficient condi- 


Qe 
tions for >, (— 1)" oe a to be nonabsolutely convergent. 


For any convergent series }°°°_, a, = S, we may define Ey as in (84) above: 


N 
Ey =|S— 5° anl. 
n=1 


Then because the series converges to S, limy_,.. Hy = 0, and conversely: in other 
words, to say that the error goes to 0 is a rephrasing of the fact that the partial 
sums of the series converge to S. Each of these statements is (in the jargon of 
mathematicians working in this area) soft: we assert that a quantity approaches 0 
and N - oo, so that in theory, given any « > 0, we have E'y < « for all suffuciently 
large N. But as we have by now seen many times, it is often possible to show that 
En — 0 without coming up with an explicit expression for N in terms of €. But 
this stronger statement is exactly what we have given in Theorem 11.28b): we have 
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given an explicit upper bound on Ey as a function of N. This type of statement 
is called a hard statement or an explicit error estimate: such statements tend 
to be more difficult to come by than soft statements, but also more useful to have. 
Here, as long as we can similarly make explicit how large N has to be in order for 
ay to be less than a given € > 0, we get a completely explicit error estimate and 
can use this to actually compute the sum S to arbitrary accuracy. 


co (= ynta 
n=1 n 


Example: We compute 5> to three decimal place accuracy. By The- 


orem 11.28b), it is enough to find N € Z* such that ayai = Wat < sin We may 
take N = 1000. Therefore 
oo 1000 
(-1)"*1 (-1)"*1 1 
) ) < : 
pz n a n Is 1001 


Using a software package, we find that 
n 


n=1 


= 0.6926474305598203096672310589.... 


Again, later we will show that the exact value of the sum is log 2, which my software 


package tells me is!* 


log 2 = 0.6931471805599453094172321214. 
Thus the actual error in cutting off the sum after 1000 terms is 
E000 = 0.0004997500001249997500010625033. 


It is important to remember that this and other error estimates only give upper 
bounds on the error: the true error could well be much smaller. In this case we 
were guaranteed to have an error at most ior and we see that the true error is 
about half of that. Thus the estimate for the error is reasonably accurate. 

Note well that although the error estimate of Theorem 11.28b) is very easy 
to apply, if a, tends to zero rather slowly (as in this example), it is not especially 
efficient for computations. For instance, in order to compute the true sum of the al- 
ternating harmonic series to six decimal place accuracy using this method, we would 
need to add up the first million terms: that’s a lot of calculation. (Thus please be 
assured that this is not the way that a calculator or computer would compute log 2.) 
Example: We compute pa ci) ” to six decimal place accuracy. Thus we need to 
choose N such that ay41 = wil < 107°, or equivalently such that (N+1)! > 10°. 
A little calculation shows 9! = 362,880 and 10! = 3,628, 800, so that we may take 
N =9 (but not N = 8). Therefore 


 (=1)" 1 6 
Id, a a ao 


Using a software package, we find 


9 (_4)n 
er 


= 0.3678791887125220458553791887. 


l4Veg, you should be wondering how it is computing this! More on this later. 
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In this case the exact value of the series is 
= == 0.3678794411714423215955237701 
e 


so the true error is 
Ey = 0.0000002524589202757401445814516374, 
which this time is only very slightly less than the guaranteed 


Tol = 0.0000002755731922398589065255731922. 


7.2. Dirichlet’s Test. 


What lies beyond the Alternating Series Test? We present one more result, an 
elegant (and useful) test due originally to Dirichlet.’ 


THEOREM 11.29. (Dirichlet’s Test) Let \>--_ an, 95°, bn be two infinite se- 
ries. Suppose that: 
(i) The partial sums By, = b; +...+ bn are bounded. 
(ii) The sequence apn is decreasing with limp+oo dn = 0. Then >>, anbn is con- 
vergent. 


Proor. Let M € R be such that B, = 6; +... +6, < M for alln € Zt. For 
€ > 0, choose N > 1 such that ay yz. Then forn >m=>N, 


n n n-1 


| yi arnbde| = | ye arn ( Br — Be-1)| = | Se apBy — by an41By| 
k=m k=m k=m 


k=m-1 


n-1 
=| S ( ak — aK41) \By + anBn — AmBm-1| 
k=m 


n-1 
x (s lax — onl) + |@n||Bn| + |@m||Bm—a| 


k=m 


n-1 n-1 
S M(S0 lax — an41]) + lanl + lam| = M (Sen — O41) + On +m) 
k=m k=m 


= M(am — Qn + On +m) = 2Mam < 2Man <e. 
Therefore 5>,, @nbn converges by the Cauchy criterion. 


In the preceding proof, without saying what we were doing, we used the technique 
of summation by parts. 

If we take b, = (-1)"*1, then Bany1 = 1 for all n and Bo, = 0 for all n, 
so {b,} has bounded partial sums. Applying Dirichlet’s Test with a sequence 
Gp, which decreases to 0 and with this sequence {b,}, we find that the series 
Yn Gnbn = ¥,,(-1)"ttan converges. We have recovered the Alternating Series 
Test! 

In fact Dirichlet’s Test yields the following Almost Alternating Series Test: 
let {a,,} be a sequence decreasing to 0, and for all n let b, € {+1} be a “sign se- 
quence” which is almost alternating in the sense that the sequence of partial 
sums B, = b} +...+ by is bounded. Then the series 5°, baa converges. 


15 Johann Peter Gustav Lejeune Dirichlet, 1805-1859 
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Exercise: Show that Dirichlet’s generalization of the Alternating Series Test is 
“as strong as possible” in the following sense: if {b,} is a sequence of elements, 
each +1, such that the sequence of partial sums B, = 61 +... + bn is unbounded, 
then there is a sequence a, decreasing to zero such that 7, dnb, diverges. 


Exercise: 
a) Use the trigonometric identity 


sin(n + 4) —sin(n — 5) 


2 
2sin(4) 


cosn = 


to show that the sequence B, = cosl1+...+cosn is bounded. 
b) Apply Dirichlet’s Test to show that the series )7°°_, <o converges. 
c) Show that $7°°_, ° is not absolutely convergent. 


Exercise: Show that >, ou is nonabsolutely convergent. 


Remark: Once we know about series of complex numbers and Euler’s formula 
e’” = cosx+isinz, we will be able to give a “trigonometry-free” proof of the pre- 
ceding two exercises. 


Dirichlet himself applied his test to establish the convergence of a certain class 
of series of a mixed algebraic and number-theoretic nature. The analytic proper- 
ties of these series were used to prove his celebrated theorem on prime numbers in 
arithmetic progressions. To give a sense of how influential this work has become, in 
modern terminology Dirichlet studied the analytic properties of Dirichlet series 
associated to nontrivial Dirichlet characters. For more information on this work, 
the reader may consult (for instance) [DS]. 


7.3. Cauchy Products III: A Divergent Cauchy Product. 


Let us give an example — due to Cauchy! — of a Cauchy product of two nonabsolutely 
convergent series which fails to converge. Take 77° 9 dn = 07-9 On = D9 Ci. 


y) 


The nth term in the Cauchy product is 


: 2 od 1 
C= 1)*(—1)7 : : 
2 NOTA 
Now (—1)'(—1)4 = (-1)'*7 = (-1)", so c, is equal to (—1)” times a sum of positive 
terms. Since i,j <n, : and thus each term in c, has absolute 


iia yee 
Ja? a 
n+ 1 terms, all of the same size, we find |c,,| > 1 for all n. Thus the general term 
of 57, Cn does not converge to 0, so the series diverges. 


> 
i+1? Vj+l — VnFl’ 
Since we are summing from i = 0 to n there are 


value at least ( 


7.4. Decomposition into positive and negative parts. 


For a real number r, we define its positive part 


r* = max(r,0) 
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and its negative part 

r~ =—min(r,0). 
Exercise: Let r be a real number. Show: 
a)r=rt—r. 
b) |r| =rt+r-. 


For any real series >, an we have a decomposition 
Se = soe =a 
n n n 
at least if all three series converge. Let us call 57, a7 and >>, a; the positive 


part and negative part of )>,, an. Let us now suppose that 5, an converges. By 
the Three Series Principle there are two cases: 


Case 1: Both >>, a, and 5°, az converge. Hence >>, |an| = >>,,(as + az) con- 
verges: i.e., )0,, Gn is absolutely convergent. 


Case 2: Both )*, af and 3°, a; diverge. Hence *,, |an| = >>, af + a, diverges: 
indeed, if it converged, then adding and subtracting )>,,an we would get that 
2>>,, a and 2°, a; converge, contradiction. Thus: 


PROPOSITION 11.30. If a series )*,, is absolutely convergent, both its positive 
and negative parts converge. If a series \\,, is nonabsolutely convergent, then both 
its positive and negative parts diverge. 


Exercise: Let 5°, an be a real series. 
a) Show that if $>,, af converges and 5>,, a; diverges then >, @n = —0o. 
b) Show that if )>, a, diverges and >, a, converges then )>,, an = 00. 


8. Power Series I: Power Series as Series 


8.1. Convergence of Power Series. 


Let {a,}929 be a sequence of real numbers. Then a series of the form )>° 9 dnx” 
is called a power series. Thus, for instance, if we had a, = 1 for all n we would 
get the geometric series )>~~_, x" which converges iff x € (—1,1) and has sum is. 
The nth partial sum of a power series is es a,x", a polynomial in x. One 
of the major themes of Chapter three will be to try to view power series as “infinite 
polynomials”: in particular, we will regard x as a variable and be interested in the 
propeties — continuity, differentiability, integrability, and so on — of the function 
f(x) = OP.9 anx” defined by a power series. 
However, if we want to regard the series aan a,x” as a function of x, what is 
its domain? The natural domain of a power series is the set of all values of x for 
which the series converges. Thus the basic question about power series that we will 


answer in this section is the following. 


QUESTION 11.31. For a sequence {an}? 9 of real numbers, for which values of 
x ER does the power series pee GnXx” converge? 
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There is one value of x for which the answer is trivial. Namely, if we plug in x = 0 
to our general power series, we get 


co 
SS a,0" =aj +a,-0+a.-0? =a. 
n=0 

So every power series converges at least at x = 0. 


Example 1: Consider the power series )**°_, n!z”. We apply the Ratio Test: 


n+1 

lim (nes = lim (n+ 1)|z]. 

n—0o nia” n—0o 
The last limit is 0 if ¢ = 0 and otherwise is +oo. Therefore the Ratio Test shows 
that (as we already knew!) the series converges absolutely at « = 0 and diverges 
at every nonzero x. So it is indeed possible for a power series to converge only at 
x =0. This is disappointing if we are interested in f(z) = }>,, az” as a function 
of x, since in this case it is just the function from {0} to R which sends 0 to apo. 
There is nothing interesting going on here. 


Example 2: Consider )>°°_, 2 


n 
! 


. We apply the Ratio Test: 


n=0 nt 

n+1 ! 
tii fee tee le eg, 
n—0o (n+ 1)! gn noo n+l 


So the power series converges for all x € R and defines a function f: R—- R. 


Example 3: Fix R € (0,00); consider °°.) tra”. This is a geometric series 


n=0 R” 
with geometric ratio p = %, so it converges iff |p| = |%| < 1, ie., iff x € (—R, R). 


R 
Example 4: Fix R € (0,00); consider 7°, are”. We apply the Ratio Test: 
‘ nR” ja|"*t n+1 1. |a| 
lim x _s=—, 
noo (n+ 1)R+1 |x|” noo n R R 


Therefore the series converges absolutely when |x| < R and diverges when |x| > R. 
We must look separately at the case |x| = R-i.e., when s = +R. When z = R, the 
series is the harmonic series }>, 4, hence divergent. But when « = —R, the series 


is the alternating harmonic series )7,, ate hence (nonabsolutely) convergent. So 
the power series converges for x € [—R, R). 

Example 5: Fix R € (0,00); consider °™, (" x”. We may rewrite this se- 
ries as > <p (—2)", i.e., the same as in Example 4 but with x replaced by —x 
throughout. Thus the series converges iff —x € [—R, R), ie., iff ¢ € (—R, R]. 


Example 6: Fix R € (0,00); consider >>, ape”. We apply the Ratio Test: 


i n?R” ae Ie} I mei a _ |a| 

noo (n+ 1)2R! fap Insc ln RR 
So once again the series converges absolutely when |x| < R, diverges when |z| > R, 
and we must look separately at c = +R. This time plugging in c = R gives )>,, a 
(=1)" 


37-: since 
n 


which is a convergent p-series, whereas plugging in x = —R gives >, 
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the p-series with p = 2 is convergent, the alternating p-series with p = 2 is abso- 
lutely convergent. Therefore the series converges (absolutely, in fact) on [—R, R]. 


Thus the convergence set of a power series can take any of the following forms: 


e the single point {0} = [0,0]. 

e the entire real line R = (—oo, co). 

e for any R € (0,00), an open interval (—R, R). 

e for any R € (0,00), a half-open interval [—R, R) or (—R, R] 
e for any R € (0,00), a closed interval [—R, R]. 


In each case the set of values is an interval containing 0 and with a certain radius, 
i.e., an extended real number R € [0,00) such that the series definitely converges 
for all x € (—R, R) and definitely diverges for all x outside of [—R, R]. Our goal is 
to show that this is the case for any power series. 


This goal can be approached at various degrees of sophistication. At the calcu- 
lus level, we have already said what is needed: we use the Ratio Test to see that 
the convergence set is an interval around 0 of a certain radius R. Namely, taking a 
general power series }>, a,x" and applying the Ratio Test, we find 

lim lansaa™ | = |x| lim ae 
noo lant” | NCO An 


“+1 the Ratio Test tells us that the series converges when 


|cjp < 1—iee., iff |2] < - ~— and diverges when |z|p > 1 — ie., iff |x| > a That 
is, the radius of convergence R is precisely the reciprocal of the Ratio Test limit p, 
with suitable conventions in the extreme cases, i.e. L 


=O, QR —YU: 


So if p = limn +0 


1 
»0 


So what more is there to say or do? The issue here is that we have assumed 
that lim, 5. “"“" 


exists. Although this is usually the case in simple examples of 
interest, it is certainly does not happen in general (we ask the reader to revisit §X.X 
for examples of this). This we need to take a different approach in the general case. 


LEMMA 11.32. Let A > 0 and let ¥0,, dna” be a power series. If S>,, dn A” 
converges, then >), 2” converges absolutely for all x € (—A, A). 


Proor. Let 0 < B < A. It is enough to show )°,, a,B” is absolutely con- 
vergent, for then so is }*, a@,(—B)". Since }>,,a,A” converges, a,A” — 0: by 
omitting finitely many terms, we may assume |a,,A”| < 1 for alln. Since 0 < 8 <1, 


YF lonB"| = Trlmai(G) < (4) < 


THEOREM 11.33. Let >>, Gnz” be a power series. 
a) There exists R € [0,00] such that: 
(i) For all x with |x| < R, SO, anx” converges absolutely and 
(ii) For all x with |z| > R, Xo, anx” diverges. 
b) If R=0, then the power series converges only at x = 0. 
c) If R=, the power series converges for all x € R. 
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d) If0 < R < ov, the convergence set of the power series is either (—R, R), [—R, R), 
(—R, R] or [-R, R]. 


PRooF. a) Let R be the least upper bound of the set of « > 0 such that 
y,, anx” converges. If y is such that |y| < R, then there exists A with |y| <A < 
R such that )7,, a,A” converges, so by Lemma 11.32 the power series converges 
absolutely on (— A, A), so in particular it converges absolutely at y. Thus R satisfies 
property (i). Similarly, suppose there exists y with |y| > R such that >>, any” 
converges. Then there exists A with R < A < |y| such that the power series 
converges on (—A, A), contradicting the definition of R. 

We leave the proof of parts b) through d) to the reader as a straightforward exercise. 


Exercise: Prove parts) b), c) and d) of Theorem 11.33. 


Exercise: Let S09 anx” and S\~ 4 b,x” be two power series with positive radii 
of convergence R, and Ry. Let R = min(Ra, Ry). Put cp = 5 Apbn—p. Show 
that the “formal identity” 


n=0 


is valid for all « € (—R, R). (Suggestion: use past results on Cauchy products.) 


The drawback of Theorem 11.33 is that it does not give an explicit description 
of the radius of convergence R in terms of the coefficients of the power series, as is 
the case when the ratio test limit p = limp—+oo seu exists. In order to achieve this 
in general, we need to appeal instead to the Root Test and make use of the limit 
supremum. The following elegant result is generally attributed to J.S. Hadamard,'® 
who published it in 1888 [Ha88] and included it in his 1892 PhD thesis. This seems 
remarkably late in the day for a result which is so closely linked to (Cauchy’s) Root 
Test. It turns out that the result was established by our most usual suspect: it was 


first proven by Cauchy in 1821 [Ca21] but apparently had been nearly forgotten. 
THEOREM 11.34. (Cauchy-Hadamard) Let 37, dnx” be a power series, and put 


O= lim sup |a;,|*. 


noo 
Then the radius of convergence of the power series is R = ae that is, the series 
converges absolutely for |x| < R and diverges for |x| > R. 
Proor. We have limsup,,_,., |anx” | = |x| limsup,_... lan|* = |a|@. Put 
R= a If |x| < R, choose A such that |x| <A < R and then A’ such that 
ee 1 
@= .<A'<-. 
R A 


Then for all sufficiently large n, lanx”|* < A’A <1, so the series converges abso- 
lutely by the Root Test. Similarly, if |z| > R, choose A such that R < |x| < A and 


then A’ such that i : 
—<A <=. 
A R 


16 Jacques Salomon Hadamard (1865-1963) 
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Then there are infinitely many non-negative integers n such that lan” | > A'A> 
1, so the series )>,, a,x” diverges: indeed a,x” ~ 0. 


Here is a useful criterion for the radius of convergence of a power series to be 1. 


COROLLARY 11.35. Let {a,}°29 be a sequence of real numbers, and let R be 
the radius of convergence of the power series et Gente: 
a) If {ay} is bounded, then R > 1. 
b) If dn + 0, then R <1. 
c) Thus if {an} is bounded but not convergent to zero, R= 1. 


Exercise: Prove Corollary 11.35. 
THEOREM 11.36. Let 7, anx” be a power series with radius of convergence R. 
Then, for any k € Z, the radius of convergence of >>, n®anx” is also R. 


PROOF. Since limy-5, (aie = limn+oo (a? = 1, by Corollary 11.22, 


n 


lim (nk)t/n = lim n*/" = 1, 
n—0oo n—-oo 


(Alternately, take logarithms and apply L’H6pital’s Rule.) Therefore 
a 


limsup(n*|ap|)" = ( lim (n'y) (iimsup nl?) = lim sup lan|*. 
n—->Co 


noo AeFOe noo 


The result now follows from the Cauchy-Hadamard Formula. 


Remark: For the reader who is less than comfortable with limits infimum and supre- 
mum, we recommend simply assuming that the Ratio Test limit o = lim,_,~ felt 
exists and proving Theorem 11.36 under that additional assumption using the Ratio 


Test. This will be good enough for most of the power series encountered in practice. 


Exercise: Let )°,, a,x" be a power series with interval of convergence J. Let J 
be the interval of convergence of )>,, nanx"—?. 

a) Show that JC I. 

b) Give an example in which J # I. 


CHAPTER 12 


Taylor Taylor Taylor Taylor 


1. Taylor Polynomials 


Recall that early on we made a distinction between polynomial expressions and 
polynomial functions. A polynomial expression is something of the form P(t) = 
ee a,t” with N € N and ao,...,ay € R. To every polynomial expression we as- 
sociated a polynomial function P(«) from R to R, given by 1 4 ee anx”. Then 
we showed that the process is reversible in the following sense: if two expressions 
Se ay,t” and uae b,t” define the same polynomial function, then a, = 6b, for 
all0 <n < N. (This came down to showing that that if a polynomial expression 
paras ant” defines the zero function, then a, = 0 for all n.) And after that — 
thankfully! — we have spoken only of polynomials. 


We wish — briefly! — to revisit this formalism in a slightly generalized context. 
Namely we want to consider polynomial c-expansions and their associated func- 
: ‘ St : N Aes 
tions. A polynomial c-expansion is a formal expression }7))_) @n(t — c)” with 
N €Nandc,a, € R. A polynomial c-expansion induces a function in an evident 
manner: we map x to peas Qn(a — c)". Now we wish to establish the following 

simple — indeed, almost trivial — but nevertheless important result. 


THEOREM 12.1. Let P be a polynomial of degree at most N, and let c € R. 


a) There is a unique polynomial c-expansion Saeme bn(t — c)” such that for 
all x €R, P(x) = 09 bn(x — cc)”. 


n=0 


b) Explicitly, we have bp = as for dlO<n<QN. 


PROOF. a) We first discuss the existence of c-expansions. Perhaps this is most 
cleanly handled by induction on the degree of P. If P is the zero polynomial or is 
a constant polynomial, then there is nothing to show: a constant a is certainly of 
the form >, bn(x — c)”: take bop = a and b, = 0 for all n > 0. Suppose now that 
N > 0 and we have established that every polynomial function of degree less than 
N has a polynomial c-expansion, and consider a degree d polynomial expression 
P(t) =ap + ait +... +an_it%—! +ant%. Then we may write 


P(t) =an(t— ©)" +Q(), 


where, since the leading term of ay(t—c)% is ayt”, Q(t) is a polynomial of degree 
less than N and thus by induction can be written as 


N-1 
Q(t) = De bn(t—c)”. 
n=0 


Then 
P(t) = bo +i (t —c) +... tby_a(t—c)% 1 +. an(t—o)%. 
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Alternately and perhaps more directly, simply write 


ag +ayt+agt?+...tayt™ = apt+a;((t—c)+c)+ao((t—c)+c)?+...tan((t-c)+0)% 


and use the binomial theorem to expand each ((t — c) + c)” in powers of t —c. 

As for the uniqueness, suppose that we have two c-expansions aoe An(t—c)”, 
Sales b,(t —c)” which induce the same function. By an argument almost identical 
to that just given, we may write eee —b,)(t—c¢)” as <s d,t”, with highest 
degree term dy = ay — by. Since the corresponding function x > yee dyx” is 
identically zero, by what we showed about polynomial functions we have d, = 0 
for all n. In particular 0 = dy = ay — by, so that years — b,)(t-—c)” = 
Rise (an — bn)(t — ¢)” = 2 d,t”. Reasoning as above we find that dy_1 = 
an—1 — by_-1 = 0, and so forth: continuing in this way we find that a, = by, for all 
0#n<N, and thus the c-expansion is unique. 

b) Consider the identity 


N 
(85) P(x) = So bp(w — ©)” = by + bi (a — c) + bo(w@—c)? +... + bn (e— 0). 
n=0 


Evaluating (85) at « =c gives P(c) = bo. Differentiating (85) gives 
P' (x) = by + 2bo(a — c) + 3b3(z — c)? +... + Nbnw(a—c)N—}, 


and evaluating at x = c gives P’(c) = by. Differentiating (85) again and evaluating 
Pp" (c) 
2 


that for 0<k < N, P}(c) = klby, so by = POO. 


at x = c gives P”(c) = 2bo, so bg = . Continuing in this manner one finds 


COROLLARY 12.2. Let I be an interval and c an interior point of I, and let 
f:IT-R be a function which is N times differentiable at c. Let 


Then Ty (a) is the unique polynomial function of degree at most N such that 
Ty (c) = f(C) for allO0<k<N. 


—-ypr £0 pee 
Proor. Applying Theorem 0 to Ty (2) k=0 ET (# — €)” gives 


(ks) 

a) , hence re (c) = f)(c) for all0 < k < N. As for the uniqueness: let P(x) be 
any polynomial of degree at most N such that P()(c) = f)(c) for 0 < k < N, and 
let Q = Tn —P. Then Q is a polynomial of degree at most N such that Q)(c) = 0 
for 0 < k < N; applying Theorem 12.1 we get Q(x) = ne ao (2 —c)F = 
4 0: (a —c)* =0, ie., Q is the zero polynomial and thus P = Ty. 


pied hem 
k! 


By definition, Ty (a) is the degree N Taylor polynomial of f at c. 


2. Taylor’s Theorem Without Remainder 
For n € N and c € I°, we say two functions f,g : J > R agree to order n at c if 


fim £02) 9) 


rc (x _ c)” 


= 0. 
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EXERCISE 12.1. Show: ifm <n and f and g agree to order n atc, then f and 
g agree to order m at c. 


EXAMPLE 12.1. We claim that two continuous functions f and g agree to order 
0 at c if and only if f(c) = g(c). Indeed, suppose that f and g agree to order 0 at 
c. Since f and g are continuous, we have 


1 SO) ie (Sa SHOR EO: 


xm—->C (x — c)® xr—->Cc 


0O= 


The converse, that if f(c) = g(c) then lim,+- f(x) — g(x) = 0, ts equally clear. 


EXAMPLE 12.2. We claim that two differentiable functions f and g agree to 
order 1 at c if and only if f(c) = g(c) and f’(c) = g'(c). Indeed, by the exercise 
above both hypotheses imply f(c) = g(c), so we may assume that, and then we find 

soy LO) 92) — app, Fa) FC) _ ax) — a6) 


mac uZ—-C @a+e xL—C t—C 


= f'(o) —9(0). 
Thus assuming f(c) = g(c), f and g agree to order 1 atc if and only f'(c) = g'(c). 


The following result gives the expected generalization of these two examples. It 
is generally attributed to Taylor,! probably correctly, although special cases were 
known to earlier mathematicians. Note that Taylor’s Theorem often refers to a 
later result (Theorem 12.4) that we call “Taylor’s Theorem With Remainder,” even 
though it is Theorem 12.3 (only) that was proved by Brook Taylor. 


THEOREM 12.3 (Taylor). Letn € N and f,g: I > R be two n times differen- 

tiable functions. Let c be an interior point of I. The following are equivalent: 
(i) We have f(c) = g(c), f'(c) =9'(0),.--, f (CE) = gC). 
(ii) The functions f and g agree to order n atc. 

Proor. Set h(x) = f(x) — g(x). Then (i) holds iff h(c) = h’(c) =... = 
A‘) (c) = 0 and (ii) holds iff lim, .. ats = 0. So we may work with h instead of 
f and g. Since we dealt with n = 0 and n = 1 above, we may assume n > 2. 

(i) = (ii): L=lim,-,. eos is of the form 2, so L’Hépital’s Rule gives 
h’ 
pie 
re n(a —c)r-! 


provided the latter limit exists. By our assumptions, this latter limit is still of the 
form 8, so we may apply L’Ho6pital’s Rule again. We do so iff n > 2. In general, 
we apply L’Hopital’s Rule n — 1 times, getting 


(n—1) (n—1) _ p(n—-1) 
L=lim h (x) 1 (ui h (x) —h a). 


coe nl(a—c) nl! 


xr—->C xr—C 


provided the latter limit exists. But the expression in parentheses is nothing else 
than the derivative of the function h("~) (x) at x =c¢ —ie., it is h(™(c) = 0 (and, 
in particular the limit exists; only now have the n — 1 applications of L’Hopital’s 
Rule been unconditionally justified), so L = 0. Thus (ii) holds. 

(ii) => (i): Let T,,(a”) be the degree N Taylor polynomial to h at c. By Corollary 


lBrook Taylor, 1685 - 1731 
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12.2, f and T,, agree to order n at c, so by the just proved implication (i) ==> (ii), 
h(a) and T,,(#) agree to order n at « =c: 
h(a) — Ty 
iin, BOK O) _6, 
wc (a _ c)” 
Moreover, by assumption h(a) agrees to order n with the zero function: 


h(x) 


a (a _ c)” as 
Subtracting these limits gives 
(86) 
eo (n) fs 
_ Tala) _ A(e) +h'(c)(e-0) +t 2-0? +...4 O(a - 0)” 
lim = lim ae =0. 
LC (a = c)” zc (x = cy” 


n 


Clearly lim,4-Tn(z) = Tn(c), so if T,,(c) € 0, then lim,_,. ae would not exist, 
so we must have T,,(c) = h(c) = 0. Therefore 


n() + *@-o) +...4 2 O@-9rt _ 


ac ("~—c)™ x4 (a —c)r-1 
As above, we have the limit of a quotient of continuous functions which we know 
exists such that the denominator approaches 0, so the numerator must also approach 
zero (otherwise the limit would be infinite): evaluating the numerator at c gives 
h'(c) = 0. And so forth: continuing in this way we find that the existence of the 


limit in (86) implies that h(c) = h’/(c) =... = h("~))(c) = 0, so (86) simplifies to 
Mie n n 
(aioe a 2, 
ae (x—c)” n! 


so h\™(c) = 0. 


Above we avoided a subtle pitfall: we applied L’H6pital’s Rule n — 1 times to 
limz—+e ee but the final limit we got was still of the form 8 —so why not apply 


L’H6pital one more time? The answer is if we do we get that 


A (a) 


? 


L= lim 


@m->Cc n! 


assuming this limit exists. But to assume this last limit exists and is equal to A (0) 
is to assume that nth derivative of h is continuous at zero, which is slightly more 
than we want (or need) to assume. 


For n € N, a function f : J + R vanishes to order n at c if lim,_,, se =:0; 


Note that this concept came up prominently in the proof of Theorem 12.3 in the 
form: f and g agree to order n at c iff f — g vanishes to order n at c. 


EXERCISE 12.2. Let f be a function which is n times differentiable at x = c, 
and let T,, be its degree n Taylor polynomial at x = c. Show that f — T,, vanishes 
to ordern atx =c. (This is just driving home a key point of the proof of Theorem 
12.3 in our new terminology.) 


EXERCISE 12.3. a) Show that for a function f : I > R, the following are 
equivalent: 


3. TAYLOR’S THEOREM WITH REMAINDER 269 


(i) The function f is differentiable at c. 
(ii) We may write f(x) = a9 + ai(@ — c) + g(x) for a function g(x) 
vanishing to order 1 at c. 

b) Show that if the equivalent conditions of part a) are satisfied, then we 
must have ag = f(c), a1 = f’(c) and thus the expression of a function 
differentiable at c as the sum of a linear function and a function vanishing 
to first order at c is unique. 


We thank Didier Piau for a correction that led to the following exercise. 


EXERCISE 12.4. Let a,b € Z*, and consider the following function fa,» :R— 
R: 


x sin 4, ifx #0 
fool) = { 0 i ifr =0 
a) Show that fa,» vanishes to order a—1 at 0 but does not vanish to order a at 0. 
b) Show that fa» is differentiable at x = 0 iff a > 2, in which case f* ,(0) =0. 
c) Show that fa» is twice differentiable atx =0 iffa>b+3. 
d) Deduce that for any n > 2, fn n vanishes to order n at x = 0 but is not twice 
differentiable at x = 0. 


3. Taylor’s Theorem With Remainder 


To state the following theorem, it will be convenient to make a convention: real 
numbers a, b, by |[a, b]| we will mean the interval [a, b] if a < b and the interval [}, a] 
if b < a. So |[a, 6]| is the set of real numbers lying between a and b. 


For a function f : |[a,]| + R, we put ||f|| = supzejaoj |f(x)I- 


THEOREM 12.4. (Taylor’s Theorem With Remainder) Letn €N, let I be an 
interval, and let f : I + R be defined andn+1 times differentiable. Let c € I° and 
x € I. Moreover, let 


"ff (e)(a — c)* 
ale) = 57 LOOe=9) 


k! 
k=0 
be the degree n Taylor polynomial of f atc, and let 
Rp (a) = f(a) — Tr(@) 


be the remainder. Then: 
a) There is z € |[c,x]| such that 


(nt1) (4 
(87) R,(x) = — a (x — z)"(a — Cc). 


b) There exists z € |[c,z]| such that 


(n+l) (y 
(88) R,(«) = = —c)"t, 


c) If f+ is integrable on |[c, a]|, then 


=f fl a@)(2- — 


270 12. TAYLOR TAYLOR TAYLOR TAYLOR 


d) We have 


|Rn(x)| = |F(2) — Tr(@)| S ee le Sel 
~ (n+1)! 
PRoor. We closely follow [S, Thm. 20.4]. 
a) First note that, notwithstanding the notation chosen, the expression R,(x) = 
f(x) — T,(«) certainly depends upon the “expansion point” c € I°: thus it would 
be more accurate to write it as R,,-(x). In fact this opens the door to a key idea: 
for any t € I°, we may consider the Taylor polynomial T;,,, to f centered at t and 
thus the remainder function 


(89) Rna(a) = f(@) — Tha(x) = f(a) - oS kl (a - ie 


To emphasize the dependence on t, we define a new function S : |[c,z]| — R by 
S(t) = Rnz(x). At first thought (my first thought, at least) consideration of S(t) 
just seems to make things that much more complicated. But in fact the assumed 
differentiability properties of f easily translate into differentiability properties of 
S(t), and these can used to our advantage in a rather beautiful way. Indeed, since 
f isn+1 times differentiable on I, S is differentiable on |[c, z]|. Differentiating 
both sides of (89) with respect to t gives 


n (k+1) (k) (n+1) 
s() = -F0-) (Ae 08 + Ge - yt) - ey. 
k=1 


We apply the Mean Value Theorem to S on |[c,x]|: there is z € |(c,z)| such that 


S(x) — S(e F —fTD(z 7 
= 2 = si) = oe : 
Noting that S(#) = Rnjz(x) = 0 and S(c) = Rn (x) = Rn(2), this gives 
peer le) 
R,(«) = S(c) — S(a) = al (x — z)"(a@— Cc). 


b) Apply the Cauchy Mean Value Theorem to S(t) and g(t) = (a —t)"*+ on |[c, a]: 
there is z € |(c,z)| such that 


Ry (2) S(e)—S(c)  S"(z)_ EP @(e- 2) f(z) 


n 


(e@—o)¥1 ~ G(x) gle) g(2) (nF (@—z)” (nF 2)!’ 
ms FON) ay 
fe epee 


c) If f+ is integrable on |[c, 2]|, then 


v ® ¢(n+1) ge —t)” 
Ra(z) = ~(S(@) - (0) =- f sy = | flr Ol t)ndt 


d) This follows almost immediately from part b); the proof is left to the reader. 


Exercise: Show that Theorem 12.4 (Taylor’s Theorem With Remainder) implies 
Theorem 12.3 (Taylor’s Theorem) under the additional hypothesis that f("* exists 
and is continuous? on the interval |[c, x]]. 


2Thanks to Nick Fink for pointing out the hypothesis of continuity seems to be needed here. 
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A. Taylor Series 


4.1. The Taylor Series. Let f : J — R be an infinitely differentiable func- 
tion, and let c € I. We define the Taylor series of f at c to be 


SST Oe@aor 
n=0 
Thus T(a) = limn—+oo Tn(x), where T;, is the degree n Taylor polynomial at c. In 
particular T(x) is a power series, so all of our prior work on power series applies. 


Just as with power series, it is no real loss of generality to assume that c = 0, 
in which case our series takes the simpler form 


2 Fm) (Q) nn 
y= Ds oe 


indeed, to get from this to the general case one merely has to make the change of 
variables x +> x —c. It is traditional to call Taylor series centered around c = 0 
Maclaurin series. But I know no good reason for this — Taylor series were in- 
troduced by Taylor in 1721, whereas Colin Maclaurin’s Theory of fluzions was not 
published until 1742 and makes explicit attribution is made to Taylor’s work.? Us- 
ing separate names for Taylor series centered at 0 and Taylor series centered at c 
often suggests — misleadingly! — to students that there is some conceptual difference 
between the two cases. So we will not use the term “Maclaurin series” here. 


Exercise: Define a function f : R > R by f(x) = ew fora # 0 and f(0) = 0. Show 
that f is infinitely differentiable and in fact f(”)(0) = 0 for all n EN. 


When dealing with Taylor series there are two main issues. 


QUESTION 12.5. Let f : I > R be an infinitely differentiable function and 


°° ™)(0)a™ : . 
T(x) = oho ioe be its Taylor series. 
a) For which values of x does T(x) converge? 


b) If for x € I, T(x) converges, do we have T(x) = f(x)? 


Notice that Question 12.5a) is simply asking for which values of x € R a power se- 
ries is convergent, a question to which we worked out a very satisfactory answer in 
§X.X. Namely, the set of values x on which a power series converges is an interval 
of radius R € [0, co] centered at 0. More precisely, in theory the value of R is given 
by Hadamard’s Formula z = limsup,,_,., lan|*, and in practice we expect to be 
able to apply the Ratio Test (or, if necessary, the Root Test) to compute R. 
If R= 0 then T(x) only converges at x = 0 and we have T(0) = f(0): this is a 
trivial case. Henceforth we assume that R € (0, 00] so that f converges (at least) 
n (—R,R). Fix a number A, 0 < A < R such that (—A, A) C I. We may then 
move on to Question 12.5b): must f(x) = T(x) for all x € (—A, A)? 


The answer is no: consider the function f(x) of Exercise X.X. f(x) is infin- 
itely differentiable and has f‘")(0) = 0 for all n € N, so its Taylor series is 


3Special cases of the Taylor series concept were well known to Newton and Gregory in the 
17th century and to the Indian mathematician Madhava of Sangamagrama in the 14th century. 
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T(e) =>)", & = ye O = 0, ie., it converges for all x € R to the zero func- 


n=0 n! 
tion. Of course f(0) = 0, but for z £0, f(x) = ex #0. Therefore f(x) # T(x) in 
any open interval around x = 0. 


There are plenty of other examples. Indeed, in a sense that we will not try to make 
precise here, “most” infinitely differentiable functions f : R — R are not equal to 
their Taylor series expansions in any open interval about any point. That’s the bad 
news. However, one could interpret this to mean that we are not really interested 
in “most” infinitely differentiable functions: the special functions one meets in 
calculus, advanced calculus, physics, engineering and analytic number theory are 
almost invariably equal to their Taylor series expansions, at least in some small 
interval around any given point x = c in the domain. 


4.2. Easy Examples. 


In any case, if we wish to try to show that a T(x) = f(a) on some interval 
(—A, A), we have a tool for this: Taylor’s Theorem With Remainder. Indeed, 
since R,(x) = |f(x) — T,(x)|, we have 


f(a) =T(e) > fle) = lim Talo) 


—> Jim | f(@) — Tr(a)| =0— Jim Rn(x) = 0. 


So it comes down to being able to give upper bounds on R,,() which tend to zero as 
n — co. According to Taylor’s Theorem with Remainder, this will hold whenever 
we can show that the norm of the nth derivative || ‘|| does not grow too rapidly. 


Example: We claim that for all « € R, the function f(x) = e* is equal to its 
Taylor series expansion at x = 0: 


First we compute the Taylor series expansion: f()(0) = f(0) = e® = 1, and f’(x) = 
e”, hence every derivative of e” is just e” again. We conclude that f()(0) = 1 for 
all n and thus the Taylor series is pee a as claimed. Next note that this power 
series converges for all real x, as we have already seen: just apply the Ratio Test. 
Finally, we use Taylor’s Theorem with Remainder to show that R,(x) — 0 for each 
fixed x € R. Indeed, Theorem 12.4 gives us 

Fadel | 
(n+1)! 


where || f+!) || is the supremum of the the absolute value of the (n+1)st derivative 
on the interval |{0,2]|. But — lucky us — in this case f(t) (x) = e® for all n and 
the maximum value of e” on this interval is e” if x > 0 and 1 otherwise, so in either 


way |[f@t? || < ell. So 
n+l 
eglel( = | 
Hales (a) 


And now we win: the factor inside the parentheses approaches zero with n and is 
being multiplied by a quantity which is independent of n, so R,(a) + 0. In fact a 


n+l 


’ 


R,(2) < 


AG: 
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moment’s thought shows that R,(a) + 0 uniformly on any bounded interval, say 
on [—A, A], and thus our work on the general properties of uniform convergence of 
power series (in particular the M/-test) is not needed here: everything comes from 
Taylor’s Theorem With Remainder. 

Example continued: we use Taylor’s Theorem With Remainder to compute e = e! 
accurate to 10 decimal places. 


A little thought shows that the work we did for f(x) = e® carries over verbatim 
under somewhat more general hypotheses. 


THEOREM 12.6. Let f(x): R— R be a smooth function. Suppose that for all 
A € [0, 00) there exists a number Ma such that for all x € [—A, A] and alln EN, 


[f™ («)| < Ma. 


a) The Taylor series T(x) = >>, eae converges absolutely for allx € R. 


b) For alla € R we have f(x) = T(x): that is, f is equal to its Taylor series 
expansion at 0. 


Exercise: Prove Theorem 12.6. 


Exercise: Suppose f : R > R is a smooth function with periodic derivatives: 
there exists some k € Z*+ such that f = f™). Show that f satisfies the hypothesis 
of Theorem 12.6 and therefore is equal to its Taylor series expansion at x = c. 


4.3. The Binomial Series. 


Even for familiar, elementary functions, using Theorem 12.4 to show R,,(x) > 0 
may require nonroutine work. We give a case study: the binomial series. 


Let a ER. For x € (—1,1), we define 
iw@=0+e2 


Case 1: Suppose a € N. Then f is just a polynomial; in particular f is defined and 
infinitely differentiable for all real numbers. 

Case 2: Suppose a is positive but not an integer. Depending on the value of a, f 
may or may not be defined for x < —1 (e.g. it is for a = 2 and it is not for a= 3), 
but in any case f is only (a) times differentiable at « = —1. 


Case 3: Suppose a < 0. Then lim,_,_1+ f(#) = oo. 


The upshot of this discussion is that if a is not a positive integer, then f is defined 
and infinitely differentiable on (—1,0o) and on no larger interval than this. 


For n € Zt, f(x) = (a)(a — 1)--- (a — (n — 1))\(1 + 2)*-”, so f™(0) = 
(a)(a —1)---(a—(n—1)). Of course we have f(0) = f(0) = 1, so the Taylor 
series to f at c= 0 is 


rey 14 Fo MOA l= =D) 
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If a € N, we recognize the nth Taylor series coefficient as the binomial coefficient 
(Cy, and this ought not to be surprising because for a € N, expanding out T(x) 
simply gives the binomial theorem: 


Va EN, (l+2)%= Ss GE 


n=0 


So let’s extend our definition of binomial coefficients: for any a € R, put 


nea (2) = @le=D-fe=a=9) 


Exercise: For any a € R,n € Z*, show 


0 Cees aa 


Finally, we rename the Taylor series to f(x) as the binomial series 


co 
Bia, x) = a ge. 
(=> (% 
The binomial series is as old as calculus itself, having been studied by Newton in the 
17th century.* It remains one of the most important and useful of all power series. 
For us, our order of business is the usual one when given a Taylor series: first, for 
each fixed a we wish to find the interval J on which the series B(a, a) converges. 
Second, we would like to show — if possible! — that for all x € I, B(a,x) = (1+2)*. 


THEOREM 12.7. Leta € R\N, and consider the binomial series 


B = ea Os 
w= 2 (=i) 
a) For all such a, the radius of convergence of B(a,x) = 1. 

b) For all a > 0, the series B(a,1) and B(a,—1) are absolutely convergent. 
c) If a € (-1,0), the series B(a,1) is nonabsolutely convergent. 

d) Ifa < —1, then B(a,—-1) and B(a,1) are divergent. 


PROOF. a) We apply the Ratio Test: 


so the radius of convergence is : =, 
b) Step 0: Let a > 1 be a real number and m € Zt. Then we have 


m 1 m 
By et ae Sai NO sf EET eh, 
a-—1l a-—l a-—l a a 


where in the first inequality we have just taken the first two terms of the usual 
(finite!) binomial expansion. Taking reciprocals, we get 


(=) a 
< ‘ 
a a+m 


4In fact it is older. For an account of the early history of the binomial series, see [C049]. 
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Step 1: Suppose a € (0,1). Choose an integer m > 2 such that 4 <a. Then 


(0), = eee cs ee) 7 iy 


n! m mn! 


m-12m-1 (n-1)m-11 1 
He =an-, 
m 2m (n-—1)m n n 


say. Using Step 0, we get 


es m 2m (n —1)m 

“nS Om =13m—1 nm—1 
— mam (n—l1)m_im-i y 11 
m-1 2-1 (n-1)m-1Inm-17 ann 


—_i_ 
a) 
nitm 


Yi(i)is aes < 


n n 


It follows that a, < --, so \(*)| < so 
fim 


This shows that B(a,1) is absolutely convergent; since |(*)(—1)"| = |(%)|, it also 
shows that B(a,—1) is absolutely convergent. 
Step 2: Using the identity (90), we find 


S(a,z) = +)" (*)o" = +) (eee, r e ») a" = (1+2)S(a—1,2). 


Thus for any fixed z, if S(a— 1,2) (absolutely) converges, so does S(a,x). By an 
evident induction argument, if S(a,x) (absolutely) converges, so does S(a +n, 2x) 
for alln € N. Since S(a, —1) and $(a, 1) are absolutely convergent for all a € (0,1), 
they are thus absolutely convergent for all non-integers a > 0. 

c) If a € (—1,0) andn EN, then 


(.0,)r()) = S7tecum 


this shows simultaneously that the sequence of terms of B(a,1) = S>~ 4 (®) is 
decreasing in absolute value and alternating in sign. Further, write a = 6 — 1, so 


that 68 € (0,1). Choose an integer m > 2 such that 4 < 6. Then 
(2) (1—6)(2—8)---(n-1-f)n-6 — 5 nap 


n (n — 1)! n n 


Arguing as in Step 1 of part b) shows that b, < —:, and hence 


nr 


lim (2) idiwie 
n 


n—-0Co n—-0co n—-0o n 


=0-1=0. 


Therefore the Alternating Series Test applies to show that S(a, 1) converges. 
d) The absolute value of the nth term of both B(a,—1) and B(a,1) is |(*)|. If 
a <—1, then |a—n| >n+1 and thus 


(21-5 


and thus (“) 4 0. By the Nth term test, S(a,—1) and S(a, 1) diverge. 
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Exercise*: Show that for a € (—1,0), the binomial series B(a, —1) diverges. 
Remark: As the reader has surely noted, the convergence of the binomial series 


S(a,x) at « = +1 is a rather delicate and tricky enterprise. In fact most texts at 
this level — even [S] — do not treat it. We have taken Step 1 of part b) from [Ho66]. 


Remark: There is an extension of the Ratio Test due to J.L. Raabe which sim- 
plifies much the of the above analysis, including the preceding exercise. 


THEOREM 12.8. Let a € R\N; let f(z) = (1+ 2)%, and consider its Taylor 
series at zero, the binomial series 


a) For all x € (—1,1), f 
b) Ifa>—1, fl) = Bl 
c) Ifa>0, f(-1) = Bl 


Proor. [La] Let T,,-1(x) be the (n — 1)st Taylor polynomial for f at 0, so 


(x) = Bla, 2). 
1 


B(a,x) = lim T,-1(2) 
nN—- Co 


is the Taylor series expansion of f at zero. As usual, put R,-1(x) = f(x) -Tn_1(2). 
a) By Theorem 12.4b), 


f"(t)(a — t)"~1dt 1 x | ee ers 
n= fe (n — 1)! =a | a(a—1)--+(a—n+1)(1+t) (a—t)”~ “dt. 


By the Mean Value Theorem for Integrals, there is @ € (0,1) such that 


a(a—1)---(a—n+1) 


Ry—1(#) = en (1 + 6x)°-" (x — Ox)" (a — 0). 
Put 
= a7 CAlel (ese = 
Then 
(14s)! = S cn(8) 
and - 


Rn-1(@) = Cn(at)ax(1 + 6x) 


Since a € (—1,1), we have t € (0,1), so |at| < 1. It follows that 377°, en(at) 
converges, so by the nth term test c,,(at) > 0 as n > oo and thus Rp_1(xz) > 0. 
b) The above argument works verbatim if z = 1 and a > —1. 

c) Ifa > 0, then by Theorem 12.7b), S(a, —1) is convergent. Moreover, a—1 > —1, 
so S772, en(1) converges and thus c,(1) > 0. But Jen(—1)| = |en(1)|, so also 
Cn(—1) > 0 and thus R,-1(-—1) > 0. 
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5. Hermite Interpolation 


It is a well-known fact that “two points determine a line”. One version of this 
is: given real numbers x, < x2 and real numbers f;, fz, there is a unique linear 
function f : RR — R such that f(21) = fi and f(#2) = fo. In a similar way, “three 
points determine a parabola”: given real numbers x; < x2 < x3 and real numbers 
fi, fo, fg, there is a unique quadratic polynomial P(x) = ax? + bx +c such that 
P(«;) = f; for i = 1,2,3. We will give a generalization. 


We work with respect to a fixed sequence 


TeH] U1 Lo Ss al SEH Sess 


of real numbers. To include this dependence explicitly in our notation would be 
cumbersome, so we will leave it implicit. 


We define a sequence {h,,}°°, of polynomials, as follows: ho = 1 and for all n € Z*, 
hn(t) = (- 21) +++ (t — fp). 

For polynomials a(t), b(t) and a nonzero polynomial c(t), we write a(t) = b(t) 

(mod c(t)) if there is a polynomial q(t) such that a(t) —b(t) = q(t)c(t). Equivalently, 

a(t) and b(t) leave the same remainder upon division by c(t). 


EXERCISE 12.5. Suppose a(t), b(t) are polynomials and c1(t) | c2(t) are nonzero 
polynomials. Show: if a = b (mod cg), then a = b (mod cy). 


EXERCISE 12.6. Suppose a(t), b(t), c(t) are polynomials with deg(a),deg(b) < 
deg(c) anda = b (mod c). Show: a= b. 

EXERCISE 12.7. Let P;(t),...,Py(t) be nonzero polynomials such that deg(P,) < 
... <deg(P,,). Suppose that A, Pi (t)+...+AnP,(t) =0 (i.e., the zero polynomial). 
Show that Ay =...= An =0. 

THEOREM 12.9. Let P(t) be a polynomial, and letn € Zt. 

a) There are unique real numbers Ao,...,An such that 
b) Suppose n = deg(P). Then there are unique real numbers Ao,...,An such that 
P(t) = Ap + Ahi (t) +... + Anhn(t). 

PROOF. a) Existence: After replacing P(t) by its remainder upon division by 
An+i(t), we may assume deg(P) < deg(hn+1), i-e., deg(P) <n. Now divide P by 
An, getting q(t) and r(t) such that P(t) = q(t)hn(t) + r(t) and deg(r) < deg(h,) = 
n. In fact, because we are dividing a polynomial P(t) of degree at most n by a 


Ys oo 


polynomial h.,(t) of degree n, the quotient q(t) is a constant, i.e., a real number: 
call it A,. Then P(t)—A,hy is a polynomial of degree at most n—1, and we divide 
it by h,-1. Continuing in this way, we arrive at the desired result. 


Uniquness: If also P(t) = Bo + Bih(t) +...+ Brh(t) (mod hy41(t)), then 
An+i(t) | (Ao — Bo) +... + (An — Bn)hn(t). 

Since the left hand side has degree n+ 1 and the right hand side has degree n, this 

means that (Ap — Bo) + (Ai — Bi)hi+...+ (An — Bn)hn(t) is the zero polynomial, 


contradicting Exercise X.X. 
b) This follows from part a) and Exercise X.X. 
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We had better hasten to explain why we care about whether two polynomials are 
congruent modulo h,,(t): what is the significance of this? The answer is that if 
f = 4 (mod h,(t)) then g interpolates f in a precise sense. In particular, this 
implies that f(#;) = g(a;) for all 1 <i <n. Indeed, f = g (mod h,) means there 
is a polynomial g such that f — g = ghn(t) = q][;_,(t— 2), and plugging in t = 2; 
gives f(x) — g(x) = 0. 


Next I claim that in the special case in which v7, < 2,41 for all n (“Lagrange 
Interpolation” ), conversely if f(#;) = g(a;) for alll <i <n then f =g (mod h,). 
Indeed, f — g vanishes at 71, so by the Root Factor Theorem is divisible by t — x1: 


f(t) — g(t) = a(t)(t — 21). 


Now evaluate the above equation at t = x2: 


0 = f(#1) — g(@2) = a1 (#2) (#2 — 1). 
Since x2 > 21, we find that qi(a2) = 0, so by Root Factor we may write qi(t) = 
g2(t)(t— x2) and thus f(t) — g(t) = (t—21)(t— %2)qo(t). And similarly we find that 
q2(%3) = 0, and so forth: in the end we get f(t) — g(t) = (t—%1)---(t-2n)an(t) = 
An(t)qn(t), so f =g (mod h,(t)). 


However, whenever ¢,-1 = Yn, the condition f = g (mod h,,) is strictly stronger 
than f(x;) = g(ai) for all 1 <i <n: the idea is that f = g (mod h,) always 
imposes “n independent linear conditions”, whereas when 2,_; = Zp then the 
constraints f(@%p—1) = g(@n-1) and f(x%,) = g(vn) are redundant. For a simple 
example, suppose 71 = %2 = 0. Then hy = ¢?, and if two polynomials have the 
same constant term, then their diference need not be divisible by t?. 


Well, let’s explore a bit more: suppose f = g (mod #?), ie., f(t) — g(t) = r(t)t?. 
The fundamental observation here is that not only do we have f (0) = g(0) but also 
f’(0) = g'(0). Indeed, differentiating gives 


f(t) -—g/(t) =r (OU + 2tr(t) = t(tr't) + 2r(d), 
so plugging in t = 0 we get f’(0) = g’(0). Once we have the idea, arriving at the 
following generalization is not difficult. 

LEMMA 12.10. Let c € R and n € Z*. For polynomials f(t) and g(t), the 
following are equivalent: 

(i) We have f =g (mod (t —c)”). 
(ii) For allO0 <k <n, we have f)(c) = g)(c). 

PRooF. (i) = > (ii): We claim that if f = g (mod (t —c)"), then f’ = g’ 
(mod (¢ —c)"~1). Indeed, writing f(t) — g(t) = (t—c)"r(t) and differentiating, we 
get 

(8) — g(t) = nit 0)" r(t) + (t= 0)" Fr"(t) = (to Manr(t) +00), 
establishing the claim. From this it follows that for all0 < k < n—1, we have 
f® = g® (mod (t — c)"~*), so by Exercise X.X f) = g) (mod t — c), hence 
FH(c) = g(c). 

(ii) ==> (i): By considering the polynomial h = f — g, it is enough to show that 
if hc) = 0 for all 0 < k <n then (t—c)” | hk. By polynomial division we may 
write h = q(t)(t —c)"” + r(t) with deg(r) < n. By the just-proved implication (i) 
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= > (ii), we know that for all 0 < k < n, the kth derivative of (t — 
so (by the linearity of derivatives) for all 0 < k <n also r™)(c) = 
that, in conjunction with deg(r) < n, this implies r = 0, hence (t — c) 
suppose not, and write r = (t —c)"™s with s(c) #0. Then 


. We claim 


c)” at cis 0, 
" | h. Indeed, 


= (t— "(ms + (t- 0") 


Plugging t = c into ms + (t — c)s’, the first term is not zero and the second term 
is, so c is not a root of ms + (t — c)s’. Continuing in this manner we find that 
r= (t—c)™~? 8 with so(c) 4 0, and so forth: eventually we get that r°”)(c) 4 0. 
Since m < deg(r) <n, this is a contradiction. 


The general meaning of f = g (mod h,,) is clear: it enforces equality of not only 
the values of f and g at 21,...,%n»,, but when there are repetitions in the sequence, 
equality of some of the derivatives of f and g. A precise enunciation is unfortunately 
rather technical. Here it goes: forn € Zt and cE R, let x.(c,n) be the number of 
times the real number c occurs in the finite sequence 21,...,%n. Then: 


PROPOSITION 12.11. For polynomials f and g and n € Z*, the following are 
equivalent: 
(i) We have f =g (mod h,). 
(ii) For alle € R and allO0 <k < 2.(c,n), we have f(c) = g(c). 


PRoor. First observe that in the case 1 = ... = Zp, then x, (c,n) = n and 
this reduces to the previous result. Suppose that after removing repetitions from 
the sequence %1,...,2% we get the increasing sequence c; < ... < c,, and for 


1<i<r, let us write m; for 7.(c,n). 
(i) == (ii): Since for each 1 < i < r we have (t — ¢;)'" | h,(t), condition (i) 
implies f = g (mod (t— c¢;)’), and then the previous result gives condition (ii). 
(ii) = (i): Condition (ii) and the previous result tell us that f = g (mod (t — 
c)") for all 1 <i <r. So for starters, f —g = (t—c1)'™'hi. We may write 
hy =(t- c2) 4241 (t) with dz > 0 and j1(ce) 4 0. Then 

f—g = (t—c2)® ((t-e1)™ f(t) = (t — e2) ha), 


where (since cy 4 cp) ke(c2) 4 0. On the other hand, we can write 
(t — c2)""*q2 = f —g = (t — c2)ka(t), 
so if dz < m2 we get 
ko(t) = (t — c2)"™@qn(t), 
and plugging in t = c gives a contradiction. So dz > m2 and thus 
ee GS (t — c)™ (t = C2)" hg. 


Continuing in this manner, eventually we get that h,(t) = []}_,(t-—ci)” | f—-g. 


In summary, we have proved the following result. 


THEOREM 12.12. (Generalized Polynomial Interpolation) Let r,my,...,m, € 
Zt, and putnt+1=m,+...+m,. Let cy < ...,¢, be real numbers, and for 
each 1 <i<rand1<j <™m,, let fi; be a real number. Then there is a unique 
polynomial P of degree at most n such that for all 1 <i<randO0<j<m,—1, 
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THEOREM 12.13. (Lagrange Interpolation Formula) Letn € Z*, let xo <...< 


In be real numbers, and let yo,..-,;Yn be real numbers. For0 <j <n, define 
L— 2X. «aL — XO L-Lj-1 U-Vji4 r— 7 

e;(x) = Il oh Paes J Daly ne 

Gchen gp = SS A ER, a 


Then an explicit formula for the unique polynomial P(x) of degree at most n such 
that P(a;) = y; for all0 <i <n of Theorem ?? is 


(91) PQ] > GG): 
j=0 


Proor. Each £;(x) is a product of n+ 1—1 =n linear polynomials, so is a 
polynomial of degree n. Therefore P(x) is a polynomial of degree at most n. To 
establish (91) the key observation is that for all 1 < 7,7 <n, €;(a;) is equal to 1 
ifi = j and 0 if i 4 7. We ask the reader to just stop and think about this: the 
formula looks a little complicated at first, so it is natural to worry that it may not 
be so easy to see this...but in fact it is immediate. The equation (91) follows, also 
immediately. 


The polynomials ¢;() are called Lagrange basis polynomials. One of the merits 
of the formula is that they do not depend on the values yo,..., Yn — or, if you like, 
the function y = f(x) that we are interpolating — but only on the values z9,..., 2 
— which in the lingo of this field are often called the interpolation nodes. 


Now look back at (??): it is reminiscent of the degree n Taylor polynomial, ex- 
cept instead of taking powers of « — 29 for a fixed center x9 we are taking different 
points x;. Just brainstorming then, might it be possible to generalize Theorem ?? 
by allowing some of the x;’s to coincide, i.e., replacing the condition 19 < ... <n 
with w <a, <...< a@,? At first glance no: if for instance 71; = x2 then the 
conditions f(v1) = fi and f(x2) = f are inconsistent unless f; = f2, in which case 
they are simply redundant. But there is a way of salvaging the situation. The key 
observation is that multiple roots of a polynomial function show up as roots of the 
derivative. More precisely, if a polynomial f can be written as (a — c)'g(x) with 
g(c) £0, then 
f' = m(az — e)™"g(x) + (x — ©) g'(z) = (w — e)™"* (mg(z) + (# — e)g'(z)). 

Observe that when we plug in x = c the expression mg(c)+(c—c)g'(c) = mg(c) £ 0. 
This shows that if a polynomial f has a root of multiplicity m > 1 at a point c, 
then the polynomial f’ has a root of multiplicity m— 1 at c. 

A reasonable way of repairing “f(v1) = fi, f(re) = fo” when x; = x2 then 
is to take the second condition as a condition on the derivative of f at x2: ie., 
f'(x2) = fa. But it is probably clearer to switch to a different notation: suppose 
that we have r distinct real numbers 7; < ... < xz,, and each occurs with a 
certain multiplicity m;. Thus our list of numbers contains m, instances of 71, m2 
instances of x2, up to m, instances of x,, thus m, +...+m, entries altogether. 
Let’s put n+ 1=m,+...+m,. Here is a more general interpolation theorem. 
Let us now try to switch back to the old notation: we give ourselves n + 1 real 
numbers “with multiplicity”: ro < xy < ... < %,, a and n+ 1 real numbers 
fo,---, fn. We write the interpolation problem as above as f(x;) = fi, but with the 
understanding that when a root is repeated more than once, the further conditions 
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are conditions on the derivatives of f. In this case we claim that the interpolation 
polynomial can be taken in the same form as above: namely, there are unique real 
numbers Ag,...,A, such that 


f = f(x) = Ao t+ Ai(u% — 20) + Ao (a — 49) (@ — 41) +... + An(@ — 0) +++ (@— Bn-1). 
At the moment we will prove this by linear algebraic considerations (which is cheat- 


ing: we are not supposed to be assuming any knowledge of linear algebra in this 
text!). Namely, since we have already shown the existence of an interpolating poly- 


nomial f of degree at most n, it suffices to show that the set of polynomials 


S = {l1,2— 120, ("@— 2%)(@ — 2),...,(@ — 40) +++ (@ — Fn_1)$ 
spans the R-vector space P,, of all polynomials of degree at most n. The set S is 
linearly independent: indeed, the polynomials have distinct degrees, so a nontrivial 
linear indeendence relationship would allow us to write a nonzero polynomial as 
a linear combination of polynomials of smaller degree, which is absurd. Further, 
#S =n+1. But P, has dimension n, so S must be a basis for P,: in paticular S 
spans Py. 


Having billed Theorem 12.12 as generalizing Theorem ??, let us now call atten- 
tion to the other extreme: suppose % =... = %, =, say. Then the interpolating 
polynomial P is precisely the degree n Taylor polynomial at c to any n times dif- 
ferentiable function f with f%(c) = f; for 0 < j <n. This brings up a key 
idea in general: let J be an interval containing the points rp < ... < 2p, and let 
f :I —R be an n times differentiable function. We define the Hermite inter- 
polation polynomial P(x) to be the unique polynomial of degree at most n such 
that for allO <i <n, P(x;) = f(a): here we are using the above slightly shady 
convention that when the x;’s occur with multiplicity greater than 1, the conditions 
P(a;) = f(a;) are actually conditions on the derivatives of P and f at 2;. 


Let us define the remainder function: for x € J, 


R(a) = f(a) — Pla). 
Following [CJ], we will now give an expression for R which generalizes one form of 
Taylor’s Theorem With Remainder. We begin with one preliminary result. 


THEOREM 12.14. (Generalized Rolle’s Theorem) Let f : I + R be n times dif- 
ferentiable, and assume that f has at least n+1 roots on I, counted with multiplicity. 
Then there is € € I with f™(¢) =0. 


Exercise: Prove Theorem 12.14. 


THEOREM 12.15. (Hermite With Remainder) Let x9 < ... < & € I, and let 
f :I— R be (n+1) times differentiable. Let P be the Hermite Interpolation 
Polynomial for f. Then, for alla € I, there is ¢ € I — in fact, lying in any closed 
interval containing £,£0,..-,%n — such that 


(92) R(e) = fla) — P(e) = EE ping, 


Proor. If « = x; for some 2, then both sides of (92) are 0, so equality holds. 
We may thus assume x 4 x; for any 7. Let c € R, and consider 


K(x) = R(x) — cla — x0) +++ (@ — fp). 
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There is a unique value of c such that K(x) = 0: namely, 
R(x) 
(@ — 20) +++ (@# — 2p) 
The function kK : J — R thus vanishes at least n + 2 times on J with multiplicity, 
so by the Generalized Rolle’s Theorem there is ¢ € I such that 


0= KM VC) = fONE) — PCHY(G) — (n+ Ile = fOM(E) - (n+ lc, 


c= 


so 
FONG) 
Coe ageeaiyts 
It follows that 
(a —29)---(w— an) 


(n+ 1)! forn@). 


R(x) = 


Remark: Restricting to Taylor polynomials, our earlier argument for the existence 
of the interpolating polynomial is certainly easier: recall this consisted of simply 
writing down the answer and checking that it was correct. However this proof of 
part b) of Taylor’s Theorem with Remainder seems easier. 


Exercise: a) Let 19 <...< %p, and let m <n. Let 


P,(a) = Ap + Ai(a@ — %0) +... + An(@ — 20) +++ (@ — Bn_-1) 
be the Hermite interpolation polynomial for a function f. Show that the Hermite 
interpolation polynomial for f with respect to the approximation points 79 <... < 
Xk is 
Pm(x) = Ap + Ai(@ — 40) +... + Am(x — 20) +++ (@ — Gm_-1). 
b) Suppose 7,1 # Lp. Show that there is ¢ € [xo, x,] such that 
4: dG) 
n n! : 


c) Show that there is a sequence {¢,} taking values in I such that 
frG) 


A, = lim ———~. 
d) Suppose that f has a continuous (n + 1)st derivative. Use part c) to recover the 


formula or the nth Taylor series coefficient. 


CHAPTER 13 


Sequences and Series of Functions 


1. Pointwise Convergence 


All we have to do now is take these lies and make them true somehow. — G. Michael! 


1.1. Pointwise convergence: cautionary tales. 


Let I be an interval in the real numbers. A sequence of real functions is a 
sequence fo, f1,---,fn,---, with each f, a function from J to R. 

For us the following example is all-important: let f(x) = S°7°., ana” be a power 
series with radius of convergence R > 0. So f may be viewed as a function 
f:(-R,R) 9 R. Put fp = yo axz*, so each f, is a polynomial of degree 
at most n; therefore f,, makes sense as a function from R to R, but let us restrict 
its domain to (—R, R). Then we get a sequence of functions fo, fi,..-,fn,.-- 


As above, our stated goal is to show that the function f has many desirable prop- 
erties: it is continuous and indeed infinitely differentiable, and its derivatives and 
antiderivatives can be computed term-by-term. Since the functions f,, have all these 
properties (and more — each f,, is a polynomial), it seems like a reasonable strategy 
to define some sense in which the sequence {f;,} converges to the function f, in 
such a way that this converges process preserves the favorable properties of the f,’s. 


The previous description perhaps sounds overly complicated and mysterious, since 
in fact there is an evident sense in which the sequence of functions f, converges to 
f. Indeed, to say that x lies in the open interval (—R, R) of convergence is to say 
that the sequence f,() = >; ax2* converges to f(z). 


This leads to the following definition: if {f,}°2, is a sequence of real functions 
defined on some interval J and f : J — R is another function, we say f, converges 
to f pointwise if for all x € I, f,(x) > f(x). (We also say f is the pointwise 
limit of the sequence {f,}.) In particular the sequence of partial sums of a power 
series converges pointwise to the power series on the interval J of convergence. 


Remark: There is similarly a notion of an infinite series of functions uae ta 
and of pointwise convergence of this series to some limit function f. Indeed, as in 
the case of just one series, we just define S, = fo +...+ f, and say that 37, fn 
converges pointwise to f if the sequence S,, converges pointwise to f. 


1George Michael, 1963-2016 
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The great mathematicians of the 17th, 18th and early 19th centuries encountered 
many sequences and series of functions (again, especially power series and Taylor 
series) and often did not hesitate to assert that the pointwise limit of a sequence of 
functions having a certain nice property itself had that nice property.? The problem 
is that statements like this unfortunately need not be true! 


EXAMPLE 13.1. Define f, = x” : [0,1] > R. Clearly f,(0) = 0" = 0, so 
fr(0) + 0. For any 0 < a <1, the sequence f,(%) = x" is a geometric sequence 
with geometric ratio x, so that fr(z) > 0 for0 <a < land f,(1) 3 1. It 
follows that the sequence of functions {f,} has a pointwise limit f : [0,1] > R, 
the function which is 0 for0O < # <1 and1 ata =1. Unfortunately the limit 
function is discontinuous at x = 1, despite the fact that each of the functions 
fn are continuous (and are polynomials, so really as nice as a function can be). 
Therefore the pointwise limit of a sequence of continuous functions need 
not be continuous. 


Example 13.1 was chosen for its simplicity, not to exhibit maximum pathology. 
It is possible to construct a sequence {f,,}°°, of polynomial functions converging 
pointwise to a function f : [0,1] + R that has infinitely many discontinuities!? 


One can also find assertions in the math papers of old that if f, converges to 
f pointwise on an interval [a,b], then i fndxz > i fdz. To a modern eye, there 
are in fact two things to establish here: first that if each f,, is Riemann integrable, 
then the pointwise limit f must be Riemann integrable. And second, that if f is 
Riemann integrable, its integral is the limit of the sequence of integrals of the f,,’s. 
In fact both of these are false! 


EXAMPLE 13.2. Define a sequence {fn}PQo with common domain [0,1] as fol- 
lows. Let fo be the constant function 1. Let f; be the function which is constantly 
1 except f(0) = f(1) = 0. Let fo be the function which is equal to fi except 
f(1/2) =0. Let fz be the function which is equal to fz except f(1/3) = f(2/3) =0. 
And so forth. To get from fn to fn41 we change the value of f, at the finitely many 
rational numbers © in [0,1] from 1 to0. Thus each fn is equal to 1 except at a finite 
set of points: in particular it is bounded with only finitely many discontinuities, so 
it is Riemann integrable. The functions f, converges pointwise to a function f 
which is 1 on every irrational point of [0,1] and 0 on every rational point of [0,1]. 
Since every open interval (a,b) contains both rational and irrational numbers, the 
function f is not Riemann integrable: for any partition of [0,1] its upper sum is 
1 and its lower sum is 0. Thus a pointwise limit of Riemann integrable functions 
need not be Riemann integrable. 


EXAMPLE 13.3. We define a sequence of functions fn : [0,1] > R as follows: 
fr(0) = 0, and fr(x) = 0 for x > 1. On the interval [0, 4) the function forms a 


2This is an exaggeration. The precise definition of convergence of real sequences did not come 
until the work of Weierstrass in the latter half of the 19th century. Thus mathematicians spoke of 
functions fn “approaching” or “getting infinitely close to” a fixed function f. Exactly what they 
meant by this — and indeed, whether even they knew exactly what they meant (presumably some 
did better than others) is a matter of serious debate among historians of mathematics. 

30n the other hand, it turns out that it is not possible for a pointwise limit of continuous 
functions to be discontinuous at every point. This is a theorem of R. Baire that belongs in a more 
advanced course [CI-GT, Thm. 4.10]. 


2. UNIFORM CONVERGENCE 285 


“spike”: f(#) = 2n and the graph of f from (0,0) to (4, 2n) is a straight line, as 
is the graph of f from (+, 2n) to (4,0). In particular fp is piecewise linear hence 
continuous, hence Riemann integable, and its integral is the area of a triangle with 
base 4 and height 2n: 5 fndx = 1. On the other hand this sequence converges 


pointwise to the zero function f. So 


1 1 
lim faigos | lim fn. 

EXAMPLE 13.4. Let g: R — R be a bounded differentiable function such that 
limn+oo g(n) does not exist. (For instance, we may take g(x) = sin(3=).) For 
né€Z*, define f,(x) = one) Let M be such that |g(x)| < M for alla ER. Then 
for allx ER, |fn(ax)| < uM 50 fn converges pointwise to the function f(x) =0 and 
thus f'(x) = 0. In particular f’(1) =0. On the other hand, for any fixed nonzero 


x, fi,(x) = 22) = g'(nz), so 


lim f;(1) = lim g’(n) does not ezist. 


Thus 
lim f;,(1) # (lim fn)'(1). 


n—-oo n—-oo 


A common theme in all these examples is the interchange of limit operations: 
that is, we have some other limiting process corresponding to the condition of 
continuity, integrability, differentiability, integration or differentiation, and we are 
wondering whether it changes things to perform the limiting process on each fy, 
individually and then take the limit versus taking the limit first and then perform 
the limiting process on f. As we can see: in general it does matter! This is not 
to say that the interchange of limit operations is something to be systematically 
avoided. On the contrary, it is an essential part of the subject, and in “natural 
circumstances” the interchange of limit operations is probably valid. But we need 
to develop theorems to this effect: i.e., under some specific additional hypotheses, 
interchange of limit operations is justified. 


2. Uniform Convergence 


Most of the above pathologies vanish if we consider a stronger notion of convergence. 


Let {fn} be a sequence of functions with domain J. We say f, converges uni- 
formly to f and write f, —> f if for all « > 0, there exists N € Z+ such that for 
alln > N and all € J, |fn(x) — f(z)| <e. 


How does this definition differ from that of pointwise convergence? Let’s com- 
pare: f, > f pointwise if for all 2 € I and all ¢ > 0, there exists n € Z* such that 
for alln > N, |f,(x) — f(x)|e. The only difference is in the order of the quantifiers: 
in pointwise convergence we are first given ¢« and x and then must find an N € Z?: 
that is, the N is allowed to depend both on ¢ and the point x € J. In the defini- 
tion of uniform convergence, we are given € > 0 and must find an N € Zt which 
works simultaneously (or “uniformly”) for all « € J. Thus uniform convergence is 
a stronger condition than pointwise convergence, and in particular if f,, converges 
to f uniformly, then certainly f, converges to f pointwise. 


286 13. SEQUENCES AND SERIES OF FUNCTIONS 


PROPOSITION 13.1 (Cauchy Criterion For Uniform Convergence). Let f, : I > 
R be a sequence of functions. The following are equivalent: 
(i) We have fn + f. 
(ii) For alle > 0, there is N € Z* such that for allm,n>N and alla € TI, 
fm(x) — fn(w)| <e. 
PROOF. (i) == (ii): Let « > 0, and choose N € Zt such that for alln > N 
and all « € I we have |fn(x) — f(#)| < 5. For all m,n > N and all « € I we have 
€ 


Lfm(2) — fu(2)I < [fm(2) — £@)| + fale) -— F@L <5 +5 


(ii) ==> (i): Let e > 0, and choose N € Zt such that for all m,n > N and all 
x € D we have 


= €. 


€ 
|fm(2) > fn(2)| < 2 
Let x € D. Since f(a) > f(x), there is M, > N such that 
€ 
fiate() ~ f2)| < §. 
Then: for all n > N we have 


[Fn w) — F(@) < [fn(a) — fate (@)| + Ufa, (@) — F(@)| < 5 + 


so fn —> f. 


€ 


2 


=S€, 


2.1. Uniform Convergence and Inherited Properties. 


The following result is the most basic one fitting under the general heading “uniform 
convergence justifies the exchange of limiting operations.” 


THEOREM 13.2. Let {f,} be a sequence of functions with common domain I, 
and let c be a point of I. Suppose that for alln € Z*, limg-se fn(z) = In. Suppose 
moreover that fn > f. Then the sequence {L,} is convergent, limy+. f(x) exists 
and we have equality: 

lim L, = lim lim f,(#) = lim f(x) = lim lim f,(z). 
nm—- oo noo TC xr—->Cc «r->Ccn—->oCo 

PROOF. Step 1: We show that the sequence {L,,} is convergent. Since we don’t 
yet have a real number to show that it converges to, it is natural to try to use the 
Cauchy criterion, hence to try to bound |Lm — Ln|. Now comes the trick: for all 
x € I we have 


[Lm — Ln| < |Lm — fm(#)| + |fm (a) — fn(a)| + |fn(@) — En|; 


By the Cauchy criterion for uniform convergence, for any € > 0 there exists N € Zt 
such that for all m,n > N and all x € I we have | fm(x) — fn(«)| < 3. Moreover, 
the fact that fm(«) > Im and fn(x) — Lp give us bounds on the first and last 
terms: there exists 6 > 0 such that if 0 < |x —c| < 6 then |Ly — fn(x)| < § 
and |Lm — fm()| < §. Combining these three estimates, we find that by taking 


x €(c—06,c+6), «x £cand m,n > N, we have 


Ar geae sa  cee un er 
ee eae 
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So the sequence {L,,} is Cauchy and hence convergent, say to the real number L. 
Step 2: We show that lim,_,. f(x) = L (so in particular the limit exists!). Actually 
the argument for this is very similar to that of Step 1: 


Since L, + L and f,(x) > f(x), the first and last term will each be less than § 
for sufficiently large n. Since f,(«) + Ln, the middle term will be less than § for 
x sufficiently close to c. Overall we find that by taking x sufficiently close to (but 


not equal to) c, we get | f(a) — L| < and thus limz_,. f(#) = L. 


COROLLARY 13.3. Let fn be a sequence of continuous functions with common 
domain I and suppose that f, > f on I. Then f is continuous on I. 


Since Corollary 13.3 is easier than Theorem 13.2, we include a separate proof. 


Proor. Let x € J. We need to show that lim,_,. f(a) = f(c), thus we need to 
show that for any « > 0 there exists 6 > 0 such that for all x with |x —c| < 6 we 
have | f(x) — f(c)| < «. The idea — again! — is to trade this one quantity for three 
quantities that we have an immediate handle on by writing 


f(x) — FCO) S |F(@) — frl)| + [fn () — fx] + ln) — FOL 
By uniform convergence, there exists n € Zt such that |f(x) — fr(a)| < § for 
all x € I: in particular |f,(e) — f(e)| = |f(e) — fa(c)| < §- Further, since f,(2) 
is continuous, there exists 6 > 0 such that for all « with |x — c| < 6 we have 


|fn(x) — fn(e)| < §- Consolidating these estimates, we get 


€ € € 
- <-+it+iH=e 
@)-fOl<5+5+5=5 
EXERCISE 13.1. Consider again f(z) = x” on the interval [0,1]. We saw in 
Example 13.1 that fn converges pointwise to the discontinuous function f which is 
0 on [0,1) and 1 ata =1. 
a) Show directly from the definition that the convergence of fy to f is not 
uniform. 
b) Try to pinpoint exactly where the proof of Theorem 13.2 breaks down when 
applied to this non-uniformly convergent sequence. 


EXERCISE 13.2. Let fr : [a,b] + R be a sequence of functions. Show that the 
following are equivalent: 


(i) We have fn + f on [a,b]. 
(ii) We have fn + f on [a,b) and fn(b) + f(b). 
THEOREM 13.4. Let {f,,} be a sequence of (Riemann- Darboux) integrable func- 
tions with common domain [a,b]. Suppose that fr “yf. Then f is integrable and 


tin [n= [amto= [oF 


PROOF. Step 1: We prove the integrability of f. Fix « > 0; since f > f, there 
is N € Z* such that for all n > N and all z € [a,0], |fn(x) — f(x)| < & it follows 
that for any subinterval [c, d] C [a, 6], 


| sup(fn; le, d]) — sup(f, [e, d])| < €,|inf(fn, le, d]) — inf(f, e, d])| < €. 
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So for any partition P = {a = 2% < 41 <<... < n_1 < Ly = b} Of [a,b] andn>N, 


n-1 
IO (fn Py U(f, P) )I S ae | sup( (fros [Zn Zn+1]) a sup(f, [Tn Tn+4a]|(Li+1 _ zi)| 
1=0 
n-1 
< €(@i41 — Zi) = (D—ale, 
1=0 


and similarly, 
Since fy is integrable, by Darboux’s Criterion there is a partition P of [a,b] such 
that U(fn,P) — L(fn,P) <¢. Thus 


< (b—a)e+e+4+ (b—a)e = (2(b— a) + le. 


Since € > 0 was arbitrary, Darboux’s Criterion shows f is integrable on [a, }]. 
Step 2: If f,g : [a,b] > R are integrable and | f(x) — g(a)| < € for all x € [a, bj, then 


fs - [a= fr- as fu- g\ < (b-a)e. 


From this simple observation and Step 1 the fact that f, > f implies ee fn - iC f 
is almost immediate. The details are left to you. 


EXERCISE 13.3. It follows from Theorem 13.4 that the sequences in Examples 
2 and 8 above are not uniformly convergent. Verify this directly. 


COROLLARY 13.5. Let {fn} be a sequence of continuous functions defined on 
the interval [a,b] such that ~~) fn > f. For each n, let F, : [a,b] + R be the 
unique function with F), = f, and F,(a) = 0, and similarly let F : [a,b] > R be the 
unique function with F’ = f and F(a) =0. Then ~~, Fy + F. 


EXERCISE 13.4. Prove Corollary 13.5. 


Our next order of business is to discuss differentiation of sequences of functions. 
For this we should reconsider Example 4: let g : R > R be a bounded differentiable 
function such that limps. g(n) does not exist, and let f,(x) = see) Let M be 
such that |g(x)| < M for all R. Then for all x € R, |fn(x)| < “4, so f, +0. But 
as we saw above, lim,-,.. f/,(1) does not exist. 

Thus we have shown the following somewhat distressing fact: uniform conver- 
gence of fr to f does not imply that f!, converges. 


Well, don’t panic. What we want is true in practice; we just need suitable hypothe- 
ses. We will give a relatively simple result sufficient for our coming applications. 


THEOREM 13.6. Let {fn}921 be a sequence of functions on [a,b]. We suppose: 
(i) Each f, is continuously differentiable on [a,b], 
(ii) The functions f, converge pointwise on [a,b] to some function f, and 
(itt) The functions f/ converge uniformly on [a,b] to some function g. 
Then f is differentiable and f’ = g, or in other words 
(lim fn)’ = lim fy. 


nm—-oo n—-oo 
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U 


Proor. Let x € [a,b]. Since f’ “4 g on [a,b], certainly f’ 4 g on [a,x]. Since 
each f/ is continuous, by Corollary 13.3 g is continuous. Now applying Theorem 
13.4 and the Fundamental Theorem of Calculus we have 


foam fF fim f= im, fi, = Jim tale) — fala) = Fe) — F(a) 

Differentiating and applying the Fundamental Theorem of Calculus, we get 
9 = (f(z) - fla) =f". 

COROLLARY 13.7. Let pier fn(x) be a series of functions converging pointwise 


to f(x). Suppose that each f}, is continuously differentiable and S\~_, f},(x) “vg. 
Then f is differentiable and f' = g: 


(93) (Do fo) = Do fa: 
n=0 n=0 
EXERCISE 13.5. Prove Corollary 13.7. 


When for a series 5°, fn it holds that (7, fn)’ = >¢,, fh, we say that the series 
can be differentiated termwise or term-by-term. Thus Corollary 13.7 gives a 
condition under which a series of functions can be differentiated termwise. 


Although Theorem 13.6 (or more precisely, Corollary 13.7) will be sufficient for 
our needs, we cannot help but record the following stronger version. 


THEOREM 13.8. Let {fn} be differentiable functions on the interval [a,b] such 
that {fn(xo)} ts convergent for some xo € [a,b]. If there is g : [a,b] > R such that 
fi, 4g on [a,b], then there is f : [a,b] + R such that f, + f on [a,b] and f’ = g. 


ProoF. [R, pp.152-153] 
Step 1: Fix « > 0, and choose N € Zt such that m,n > N implies |fm(xo) — 
fn(%o)|5 and |fy,(t) — fr (| < Hoa) for all t € [a,b]. The latter inequality is 
telling us that the derivative of g := fm — fn is small on the entire interval [a,b]. 
Applying the Mean Value Theorem to g, we get ac € (a,b) such that for all 


x,t € [a,b] and all m,n>N, 


(04) ate) - 901 = be tll set (Sp) s §. 


It follows that for all x € [a, }], 
Ifm(2) — fr(x)| = lg(x)| < |g(x) — 9(20)| + lg(xo)| < ; + 5 


By the Cauchy criterion, f,, is uniformly convergent on [a,b] to some function f. 
Step 2: Now fix x € [a,b] and define 


_ falt) — fala) 


= €. 


Yn(t) fF 5 
and ; 
p(t) = HO= 1) 


so that for all n € Z*, limy— Yn(t) = fi), (x). Now by (94) we have 


lem(t) — en S aH 
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for all m,n > N, so once again by the Cauchy criterion y,, converges uniformly for 
allt £ x. Since fy, + f, we get yn + y for all t 4 x. Finally we apply Theorem 
13.2 on the interchange of limit operations: 


f'() = lim g(t) = lim lim gn(t) = lim lim yn(t) = lim f, (2). 


noo tox N—0o 


2.2. The Weierstrass M-test. 


We have just seen that uniform convergence of a sequence of functions (and possibly, 
of its derivatives) has many pleasant consequences. The next order of business is to 
give a useful general criterion for a sequence of functions to be uniformly convergent. 


For a function f : J > R, we define 
II fll = sup |f(~)|- 
vel 
In (more) words, || f|| is the least M € [0, co] such that |f(ax)| <M for all « € I. 


THEOREM 13.9. (Weierstrass M-Test) Let {f,}°9 be a sequence of functions 
defined on an interval I. Let {M,,}°2o be a non-negative sequence such that || fn|| < 
M,, for alln and M =v) Mn < 00. Then 7-4 fn is uniformly convergent. 


Proor. Let Sy(x) = Se fr(x). Since 5°,, M, < 00, for each € > 0 there is 
No € Zt such that for all N > No, nsw Mn <e. Fora el, N >No andk EN, 


N+k 
Swe) -Sw(@)l=| > fale) < Yo [fale < 0 Ma <e 
n=N+1 n>N n>N 


Therefore the series is uniformly convergent by the Cauchy criterion. 


3. Power Series II: Power Series as (Wonderful) Functions 


THEOREM 13.10. (Wonderful Properties of Power Series) Let \>~° 9 ana” be a 
power series with radius of convergence R > 0. Consider f(x) = 7 9 anv” as a 
function f :(—R,R) > R. Then: 

a) f is continuous. 
b) f is differentiable. Morever, its derivative may be computed termwise: 


f(z) = yy nage" — 
n=1 


c) Since the power series f’ has the same radius of convergence R > 0 as f, f is 
in fact infinitely differentiable. 
d) For alln EN, f™(0) = (n!)an. 


PROOF. 

a) Let 0 < A < R, so f defines a function from [—A, A] to R. We claim that 
the series )7,, a,x” converges to f uniformly on [—A, A]. Indeed, as a function 
on [—A, A], we have ||a,7”|| = |a,|A", and thus 5°, ||@nz”|| = >, |an|A” < 
oo, because power series converge absolutely on the interior of their interval of 
convergence. Thus by the Weierstrass M-test f is the uniform limit of the sequence 
Sn(x“) = ar a,z". But each S,, is a polynomial function, hence continuous and 
infinitely differentiable. So by Theorem 13.2 f is continuous on [—<A, A]. Since any 
x € (—R, R) lies in [—A, A] for some 0 < A < R, f is continuous on (—R, R). 
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b) According to Corollary 13.7, in order to show that f = )7,,anz” = SO, fn is 
differentiable and the derivative may be compuited termwise, it is enough to check 
that (i) each f,, is continuously differentiable and (ii) }7,, ff, is uniformly convergent. 
But (i) is trivial, since f, = a,x” — of course monomial functions are continuously 
differentiable. As for (ii), we compute that >, f/, = °,,(anz”) = 0, ndn-12"?. 
By X.X, this power series also has radius of convergence R, hence by the result of 
part a) it is uniformly convergent on [—A, A]. Therefore Corollary 13.7 applies to 
show 7 (4) = i, op tae 

c) We have just seen that for a power series f convergent on (—R, R), its derivative 
f’ is also given by a power series convergent on (—R,R). So we may continue in 
this way: by induction, derivatives of all orders exist. 

d) The formula f) (0) = (n!)a, is simply what one obtains by repeated termwise 
differentiation. We leave this as an exercise to the reader. 


EXERCISE 13.6. Prove Theorem 13.10d). 


EXERCISE 13.7. Show that if f(x) = Oj anz” has radius of convergence 


BO, then FG) = wee is an anti-derivative of f. 


The following exercise drives home that uniform convergence of a sequence or series 
of functions on all of R is a very strong condition, often too much to hope for. 


EXERCISE 13.8. Let {Pp : R — R}92, be a sequence of polynomials such that 
P,, + f on all of R. 
a) Show: there is N € Z* such that for alln > N we have P, — Pn is 
constant. 


(Hint: a polynomial is bounded on R iff it is constant.) 
b) Deduce that there is C € R such that f = Py+C. 


In particular, a sequence of polynomials that is uniformly convergent on R has 
eventually constant degree and the limit function is also a polynomial. 


EXERCISE 13.9. Let 30, nz” be a power series with infinite radius of conver- 
gence, hence defining a function f :R— R. Show that the following are equivalent: 


(i) The series S,, dnx” is uniformly convergent on R. 


(ii) We have a, = 0 for all sufficiently large n. 


EXERCISE 13.10. Let f(x) = S79 anx” be a power series with a, > 0 for 
all n. Suppose that the radius of convergence is 1, so that f defines a function on 
(—1,1). Show that the following are equivalent: 

(i) The series \>,, dn converges. 


(ii) The power series converges uniformly on [0, 1]. 
(iii) The function f is bounded on (0,1). 


The fact that for any power series f(x) = )>,, anx” with positive radius of conver- 
£™ (0) 


ni 

COROLLARY 13.11. (Uniqueness Theorem) Let f(x) = 30, ana” and g(x) = 
>, Onx” be two power series with radii of convergence Ra and Ry withO < Ra < Ro, 
so that both f and g are infinitely differentiable functions on (—Rq, Ra). Suppose 
that for some 6 withO <6 < Ra we have f(x) = g(x) for all x € (—6d,6). Then 
Gn = by for all n. 


gence we have ay = yields the following important result. 
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EXERCISE 13.11. Suppose f(a) =>, dnx” and g(x) = >, bnx” are two power 
series each converging on some open interval (—A, A). Let {an }92, be a sequence 
of elements of (—A, A) \ {0} such that limy_,.. 2, = 0. Suppose that f(an) = g(Zn) 
for alln € Z*. Show that an = by for all n. 


The upshot of Corollary 13.11 is that the only way that two power series can be 
equal as functions — even in some very small interval around zero — is if all of their 
coefficients are equal. This is not obvious, since in general S77? 9 dn = So7o9 bn 
does not imply a, = b, for all n. Another way of saying this is that the only power 
series a function can be equal to on a small interval around zero is its Taylor series. 


CHAPTER 14 


Serial Miscellany 


Let p € R. Recall that the p-series >>, ae converges iff p > 1. It is another 
matter entirely to determine the sum of the series exactly. In this section we 
devote our attention to the p = 2 case. 


THEOREM 14.1. (Euler) 7°, 4=% 


n=1 n= 6° 


2 


Euler’s original argument is brilliant but not fully rigorous by modern standards. 
Since then several branches of mathematical analysis have been founded which give 
systematic tools for finding sums of this and similar series. In particular if one 
learns about Fourier series or complex analysis then very natural proofs can 
be given, but both of these topics are beyond the scope of an honors calculus course. 

On the other hand, in the intervening centuries literally hundreds of proofs of 
Theorem 14.1 have been given, some of which use only tools we have developed 
(or indeed, no tools beyond standard freshman calculus). Among these we give 
here a particularly nice argument due to D. Daners [Da12] following Y. Matsuoka 
[Ma61]. In fact this argument barely uses notions from infinite series! Rather, 

ci N 1 


it gives an upper bound on 4 — Donzi zz in terms of N which approaches 0 as 


N + , and this certainly suffices. Precisely, we will show the following result. 


THEOREM 14.2. For all positive integers N, 
2 N 2 


T 1 7 
< is : 
OS 5 So aa 


The proof will exploit a family of trigonometric integrals. For n € Zt, we define 


n mn 


z 3 
A, = i cos?" dz, By, = 7 x? cos?” «dx. 
0 0 


EXERCISE 14.1. Show that An, By, >0 for alln > 1. 


LEMMA 14.3. a) For all positive integers n, we have: 


3 An Aye 
(95) | sin? x cos*"—) adx = ee aes Es 


b) For all positive integers n, we have 
(96) An = (2n — 1)nBy_1 — 2n? Bn. 
PRooF. a) Integrating by parts gives 


n 


z 
An = i cos x cos?! rdx 
0 
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m 


= he (sin x) ((2n — 1) cos?» x) (— sin x) dx 


Ona 


= (sin x)(cos?"~} 2) | 


n n 


2 2 
= (2n — yf sin? x cos*"—) adx = (2n — yf (1 — cos? x) cos?" ada 
0 0 


= (2n — 1)(An_1 — An). 
Thus 


2 A 
| sin? x cos?(""—)) adx = *_=A,-1—Apn, 
j In—1 


and 


Hing. 2 ree An \_ 1 ((2n-1)Ant+An\ An 
Qn In\o"* mM-1 Qn In —1 Qn — 1 


b) Integrating by parts twice gives! 


3 z 
A,= | 1- cos?” xdx = 2n | xsinaz cos?” ! adr 
0 0 


z 
= nf x? (cosa cos”! x — (2n — 1) sin? x cos?("-) r) dx 
0 


nm 


2 
=—-nB,+ n(2n = yf z?(1 — cos2 z) cos2("—)) xdxr 
0 


= (2n — 1)nB,_1 - Qn? Br. 


Dividing (96) by n?.A,, and using (95), we get 
i (Habe OR. DB. OB, 


n2 nAn An An— 1 An , 


Thus for all n € Zt we have 


it, - 2Bn-1 2Bn\  2Bo 2Bn 
wat 2 Be An-1 An - Ao An 
Since 
A a B : 2d ; 
0 -| ts 2 0 - | Tat = 24’ 
we have 
2Bo ont 1 
Ao 6 / 


and thus for all N € Z*, 


Equivalently 


N 
1 2B 
(97) if Ys= £0. 


x 
lThis time we leave it to the reader to check that the boundary terms uv |j evaluate to 0. 
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LEMMA 14.4. For all x € [0,5], we have: 


and thus 


2 
r< (5) sin? x. 
EXERCISE 14.2. Prove Lemma 14.4. (Hint: use convexity!) 


Using Lemma 14.4 and Lemma 14.3a) with N =n — 1 we get 


N z 
1 2B 2 2 
ge" ) = N= | ax cos’% xdax 
0 


6 a tie An An 
2 (m\? f? . » oN 2 (m7\2 An 7 
pees os : = = 
Gy | Bone ade aa) XN+1) 4(N +1)’ 


which proves Theorem 14.2. 


2. Rearrangements and Unordered Summation 


2.1. The Prospect of Rearrangement. 


In this section we systematically investigate the validity of the “commutative law” 
for infinite sums. Namely, the definition we gave for convergence of an infinite series 


aj t+agt+...+@n,+... 


in terms of the limit of the sequence of partial sums A, = a; +...+ Gy, makes 
at least apparent use of the ordering of the terms of the series. Note that this is 
somewhat surprising even from the perspective of infinite sequences: the statement 
dy, — L can be expressed as: for all € > 0, there are only finitely many terms of the 
sequence lying outside the interval (Z — ¢, £ + €), a description which makes clear 
that convergence to L will not be affected by any reordering of the terms of the 
sequence. However, if we reorder the terms {a,,} of an infinite series )°°°_, dn, the 
corresponding change in the sequence A,, of partial sums is not simply a reordering, 
as one can see by looking at very simple examples. For instance, if we reorder 


i ae! 1 
aa Sa Maa 
as 
UL seek ip ak 1 
a ae: ay 
Then the first partial sum of the new series is + whereas every nonzero partial sum 


of the original series is at least 5. 


Thus there is some evidence to fuel suspicion that reordering the terms of an infi- 
nite series may not be so innocuous an operation as for that of an infinite seuqence. 
All of this discussion is mainly justification for our setting up the “rearrangement 
problem” carefully, with a precision that might otherwise look merely pedantic. 


Namely, the formal notion of rearrangement of a series pa dy, begins with a 
permuation o of N, ie., a bijective function o : N > N. We define the rearrange- 
ment of )>° 9 dn by o to be the series 7° 9 dg(n)- 
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2.2. The Rearrangement Theorems of Weierstrass and Riemann. 


The most basic questions on rearrangements of series are as follows. 


QUESTION 14.5. Let °°.) an = S is a convergent infinite series, and let o be 
a permutation of N. Then: 
a) Does the rearranged series Sy 9 Gg(n) converge? 
b) If it does converge, does it converge to S? 


As usual, the special case in which all terms are non-negative is easiest, the case 
of absolute convergence is not much harder than that, and the case of nonabsolute 
convergence is where all the real richness and subtlety lies. 


Indeed, suppose that a, > 0 for all n. In this case the sum A = )7* 9 ay, € [0,00 
is simply the supremum of the set A, = ae a, of finite sums. More gener- 
ally, let S = {ni,...,n,} be any finite subset of the natural numbers, and put 
Ag =n, +...+4n,- Now every finite subset S C N is contained in {0,...,N} for 
some N EN, so for all S, Ag < Aw for some (indeed, for all sufficiently large) N. 
This shows that if we define 


A’ =sup Ag 
3 


as S ranges over all finite subsets of N, then A’ < A. On the other hand, for all 
NEN, An =a40+.-.+4n = Ago,....n}: in other words, each partial sum Ay 
arises as Ag for a suitable finite subset S. Therefore A < A’ and thus A = A’. 


The point here is that the description er Gn = Supg Ag is manifestly unchanged 
by rearranging the terms of the series by any permutation o: taking S ++ a(S) 
gives a bijection on the set of all finite subsets of N, and thus 


“s dy = sup Ag = sup Agcg) = a Ae(n)- 
n=0 8 8 n=0 


The case of absolutely convergent series follows rather easily from this. 


LEMMA 14.6. Let )7,, an be a real series with a, > 0 for alln and sum A € 
[(0, co]. Then A is the supremum of the set of all finite sums Ag = Yo -g Gn as S 
ranges over all nonempty finite subsets of N. 


nes 


PrRooF. Let A be the supremum of the finite sums Ag. For N € N, let 
Ayn = pe Gyn. Since a, > O for all n, the sequence Ay is increasing, so 
A = limy-.. An = supy An. Since Ay = Axo,...,.N}s we have A = supy An < 
supg Ag = A. On the other hand, for any nonempty finite subset S of N, let N be 
the largest element of S. Then S$ C {0,...,N} so As = Donegan < een an = 
An <A,so A=supAgs < A. Thus A= A. 


The point of Lemma 14.6 is that we have expressed the sum of a series with non- 
negative terms in a way which is manifestly independent of the ordering of the 
terms:? for any bijection o of N, as S = {n1,...,n,} ranges over all finite subsets 


2This is a small preview of “unordered summation”, the subject of the following section. 
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of N, so does (5) = {o(n1),...,a(mx)}. It follows that 


S> An = s Ae(n) € [0, co], 
n=0 n=0 


i.e., rearrangement of a series with non-negative terms does not disturb the conver- 
gence/divergence or the sum. 


THEOREM 14.7 (Dirichlet-Weierstrass). Let S>>- a, be an absolutely conver- 
gent series with sum A. Then for every permutation o of N, the rearranged series 
0 Go(n) converges to A. 


Proor. Put A = 0° 9 an. Fixe > 0 and let No € N be such that )°7° ny, lanl < 
e. Let Mo € N be sufficiently large so that the terms a,(9),---,@o(a)) include all 
the terms ag,...,@N —1 (and possibly others). Then for all M > Mo, 


M M co co 
[S> ao(ny — Al =| 95 dairy — Do nl < lan| < €. 
n=0 n=0 n=0 


n=No 
Indeed: by our choice of MZ we know that the terms ao,...,@N,—-1 appear in both 
oer Qg(n) and oy a, and thus get cancelled; some further terms may or may 
not be cancelled, but by applying the triangle inequality and summing the absolute 
values we get an upper bound by assuming no further cancellation. This shows 
lo) : M 
enzo Ao(n) = limm oo in=0 Go(n) = A. 


EXERCISE 14.3. a) Give a proof of Step 1 of Theorem 14.7 that bypasses 
Lemma 14.6. (Suggestion: by reasoning as in Step 2, argue that for each 
€ > 0 and all sufficiently large N, Sy x \@o(n)| < €- 

b) Use the decomposition of )~,, an, into its series of positive parts )~,, aft 
and negative parts )_,,a;,, to give a second proof of Step 2 of Theorem 


14% 


THEOREM 14.8 (Riemann Rearrangement Theorem). Let S>7° 9 a be a non- 
absolutely convergent series. For any B € [—co, ox], there exists a permutation o 
of N such that \>° 9 Go(n) = B. 


PROOF. 

Step 1: Since }7,, ay, is convergent, we have a, — 0 and thus that {a,,} is bounded, 
so we may choose M such that |a,| < M for all n. We are not going to give an 
explicit “formula” for a; rather, we are going to describe o by a certain process. 
For this it is convenient to imagine that the sequence {a,,} has been sifted into a 
disjoint union of three subsequences, the first {p,,} consisting of the positive terms, 
the second {n,,} consisting of the negative terms and the third consisting of the zero 
terms. The first two sequences must be infinite; the third may be infinite, finite or 
empty. It is rather clear that the zero terms do not affect anything — they are just 
a minor bookkeeping nuisance. So we will assume until the end of the proof that 
there are no zero terms and then address how to to deal with them. 

The key point here is Proposition 11.30 which tells us that since the convergence 
is nonabsolute, we have )7,, Pn = 00, >), %m = —0o. (This is how we know there 
are infinitely many positive terms and infinitely many negative terms.) So we may 
specify a rearangement as follows: we specify a choice of a certain number of positive 
terms, then a certain number of negative terms, then a certain number of positive 
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terms, and so on. As long as we include a finite, positive number of terms at each 
step, then in the end we will have included every term p, and n, eventually, hence 
we will get a rearrangement. 

Step 2 (diverging to oo): to get a rearrangement diverging to oo, we proceed as 
follows: we take positive terms pj, p2,... in order until we arrive at a partial sum 
which is at least + 1; then we take the first negative term n;. Since |ni| < M, 
the partial sum p; + ...+ pn, +71 is still at least 1. Then we take at least one 
more positive term py,+1 and possibly further terms until we arrive at a partial 
sum which is at least M+ 2. Then we take one more negative term ng, and note 
that the partial sum is still at least 2. And we continue in this manner: after the 
kth step we have used at least k positive terms, at least k negative terms, and all 
the partial sums from that point on will be at least k. Therefore every term gets 
included eventually and the sequence of partial sums diverges to +o. 

Step 3 (diverging to —oo): An easy adaptation of the argument of Step 2 leads to 
a permutation o such that Spear Ae(n) = —0o. We leave this case to the reader. 
Step 4 (converging to B € R): if anything, the argument is simpler in this case. We 
first take positive terms p),...,pn,, Stopping when the partial sum p; +...+ pn, 
is greater than B. (To be sure, we take at least one positive term, even if 0 > 
B.) Then we take negative terms 7,...,ny,, stopping when the partial sum 
pit...+pn, tri +...+ ny, is less than B. Then we repeat the process, taking 
enough positive terms to get a sum strictly larger than B then enough negative 
terms to get a sum strictly less than B, and so forth. Because both the positive 
and negative parts diverge, this construction can be completed. Because a, — 0, 
a little thought shows that the absolute value of the difference between the partial 
sums of the series and B approaches zero. 

Step 4: Finally, we deal with the possibility of zero terms. In every case we have 
an infinite step process where at each step we add at least one and finitely many 
positive terms followed by at least one and finitely many negative terms. To deal 
with zero terms, at the end of each step we add in one zero term if we have any 
zero terms remaining. Thus if zero occurs as a term infinitely many times, then at 
the end of each step we add a zero term, whereas if zero occurs a finite number A 
times, then we add one zero term at the end of each of the first A steps. 


The conclusion of Theorem 14.8 holds under somewhat milder hypotheses. 


EXERCISE 14.4. Let >, a, be a real series such that a, > 0, >>, a* = 00 
and \*,,a, = —oo. Show that the conclusion of Theorem 14.8 holds: for any 
A € [—0o, oo], there exists a permutation o of N such that \>° 9 4o(n) = A. 


EXERCISE 14.5. Let )>, an be a real series such that S*,, a, = oo. 


a) Suppose that the sequence {ay} is bounded. Show that there exists a per- 
mutation o of N such that >, Ag(n) = &. 

b) Does the conclusion of part a) hold without the assumption that the se- 
quence of terms is bounded? 


Theorem 14.8 exposes the dark side of nonabsolutely convergent series: just by 
changing the order of the terms, we can make the series diverge to too or converge 
to any given real number! Thus nonabsolute convergence is necessarily of a more 
delicate and less satisfactory nature than absolute convergence. With these issues 
in mind, we define a series }>,,@, to be unconditionally convergent if it is 
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convergent and every rearrangement converges to the same sum, and a series to be 
conditionally convergent if it is convergent but not unconditionally convergent. 
Then much of our last two theorems may be summarized as follows. 


THEOREM 14.9. (Main Rearrangement Theorem) A convergent real series is 
unconditionally convergent if and only if it is absolutely convergent. 


Many texts do not use the term “nonabsolutely convergent” and instead define a se- 
ries to be conditionally convergent if it is convergent but not absolutely convergent. 
Aside from the fact that this terminology can be confusing to students to whom 
this rather intricate story of rearrangements has not been told, it seems correct to 
make a distinction between the following two a priori different phenomena: 


e >), an converges but >>, |an| does not, versus 
© on Gn Converges to A but some rearrangement >7,, @¢(n) does not. 


As we have seen, these two phenomena are equivalent for real series. However 
the notion of an infinite series )7,,a,, absolute and unconditional convergence 
makes sense in other contexts, for instance’ for series with values in an infinite- 
dimensional Banach space or with values in a p-adic field. In the former case 
it is a celebrated theorem of Dvoretzky-Rogers [DR50] that there exists a series 
which is unconditionally convergent but not absolutely convergent, whereas in the 
latter case one can show that every convergent series is unconditionally convergent 
whereas there exist nonabsolutely convergent series. 


EXERCISE 14.6. Let baer dy be any nonabsolutely convergent real series, and 
let —-oo <a<A< oo. Show that there exists a permutation o of N such that the 
set of partial limits of 37° 9 dg(n) 18 the closed interval [a, A]. 


2.3. Unordered summation. 


It is very surprising that the ordering of the terms of a nonabsolutely convergent 
series affects both its convergence and its sum — it seems fair to say that this phe- 
nomenon was undreamt of by the founders of the theory of infinite series. 


Armed now, as we are, with the full understanding of the implications of our defini- 
ion of eee Gy, as the limit of a sequence of partial sums, it seems reasonable to 
ask: is there an alternative definition for the sum of an infinite series, one in which 
the ordering of the terms is a priori immaterial? 

The answer to this question is yes and is given by the theory of unordered 
summation. 


To be sure to get a definition of the sum of a series which does not depend on 
the ordering of the terms, it is helpful to work in a context in which no ordering is 
present. Namely, let S be a nonempty set, and define an S-indexed sequence of 
real numbers to be a function a, : S —+ R. The point here is that we recover the 
usual definition of a sequence by taking S = N (or S = Z*) but whereas N and Zt 
come equipped with a natural ordering, the “naked set” S does not. 


3Both of these are well beyond the scope of these notes, i.e., you are certainly not expected 
to know what I am talking about here. 
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We wish to define Doses ds, i.e., the “unordered sum” of the numbers a, as s 
ranges over all elements of S. Here it is: for every finite subset T = {s1,...,5n} 
of S$, we define ap = Sep Gs = As, +... + sy. (We also define ag = 0.) Finally, 
for A € R, we say that the unordered sum }°,-¢a; converges to A if: for all 
€ > 0, there exists a finite subset Tg C S such that for all finite subsets Tp CT CS 
we have jar — A| < ¢. If there exists A € R such that }7,.ga; = A, we say that 
discs ds is convergent or that the S-indexed sequence a, is summable. (When 
S = Z2N we already have a notion of summability, so when we need to make the 
distinction we will say unordered summable.) 


Notation: because we are often going to be considering various finite subsets T 
of a set S, we allow ourselves the following time-saving notation: for two sets A 
and B, we denote the fact that A is a finite subset of B by A Cy B. 


EXERCISE 14.7. Suppose S is finite. Show that every S-indexed sequence a. : 


S —>R is summable, with sum ag = ses As. 


EXERCISE 14.8. If S = @, there is a unique function ae: 2 > R. Convince 


yourself that the most reasonable value to assign Y) eg as 1s 0. 


EXERCISE 14.9. Give definitions for Y) eg as = 00 and Yo eg as = —00. 


Confusing Remark: We say we are doing “unordered summation”, but our se- 
quences take values in R, in which the absolute value is derived from the order 
structure. One could also consider unordered summation of S-indexed sequences 
with values in an arbtirary normed abelian group (G,| |).4 A key feature of R is 
the positive-negative part decomposition, or equivalently the fact that for M € R, 
|M| > A implies M > A or M < —A. In other words, there are exactly two 
ways for a real number to be large in absolute value: it can be very positive or very 
negative. At a certain point in the theory considerations like this must be used in or- 
der to prove the desired results, but we delay such arguments for as long as possible. 


The following result holds without using the positive-negative decomposition. 


THEOREM 14.10 (Cauchy Criterion for Unordered Summation). Leta,: SR 
be an S-index sequence, and consider the following assertions: 
(i) The S-indexed sequence a, is summable. 
(ii) For all e > 0, there exists a finite subset T. C S' such that for all finite 
subsets T,T’ of S containing Te, 


| So as— > as| <e. 
seT seET’ 
(iii) For all ¢ > 0, there exists T. Cz S such that: for all T Cy S with 
TOT, = @, we have |ar| =| di ,er 4s| < €. 
(iv) There is M €R such that for all T Cy S, lar| < M. 


PROoF. (i) => (ii) is immediate from the definition. 
(ii) ==> (i): We may choose, for each n € Z*, a finite subset T;, of S such that 
Tn C Ty4i for all n and such that for all finite subsets T,T7’ of S containing T), 


4 And it’s not totally insane to do so: these arise in functional analysis and number theory. 
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lar —ar| < §. It follows that the real sequence {az, } is Cauchy, hence convergent, 
say to A. We claim that a. is summable to A: indeed, for « > 0, choose n > 2 
Then, for any finite subset T containing T,, we have 
€ € 
lar — Al < jar —ar,| + lar, — Al < 5 56 

(ii) = > (iii): Fix « > 0, and choose 7p Cy as in the statement of (ii). Now let 
T Cy S with TAT) = @, and put T’ = TUTp. Then 7” is a finite subset of $ 
containing Tp and we may apply (ii): 


IDoasl=l >) as— Dd) asl <e. 
seT seT’ s€To 


(iii) => (ii): Fix € > 0, and let T. Cy S be such that for all finite subsets T of S 
with TNT. = @, |ar| < §. Then, for any finite subset T’ of S containing Te, 


ee 
5° 
From this and the triangle inequality it follows that if T and T”’ are two finite 


subsets containing T-, 


lar: _ aT. = lar r, 


lar —ar'| <e. 
(iii) => (iv): Using (iii), choose T; Cy S such that for all T’ Cy S with T; NT’ = 
@, |ay’| < 1. Then for any T Cy S, write T = (T\T,) U(TN7}), so 
larl<| So asl+] D5 al 1+ S5 lal, 
seT\Ty sEeTOT, sEeT, 


so we may take M=1+)> 


las|. 


seETi 


Confusing Example: Let G = Z, with its standard norm. Define a, : Zt > G 
by a, = 1 for all n. Because of the non-Archimedean nature of the norm, we have 
for any T Cy S |ar| = |#T| < 1. Therefore a, satisfies condition (iv) of Theorem 
14.10 above but not condition (iii): given any finite subset T C Zt, there exists a 
finite subset 7’, disjoint from T, such that |ar’| = 1: indeed, we may take T’ = {n}, 
where n is larger than any element of T and prime to p. 

Although we have no reasonable expectation that the reader will be able to 
make any sense of the previous example, we offer it as motivation for delaying 
the proof of the implication (iv) == (i) above, which uses the positive-negative 
decomposition in R in an essential way. 


THEOREM 14.11. An S-indexed sequence a, : S > R is summable iff the finite 
sums ae uniformly bounded: i.e., there exists M € R such that for all T Cy S, 
lar| < M. 


ProoFr. In Theorem 14.10 above we showed that if a. is summable, the finite 
sums are uniformly bounded. Now suppose a, is not summable, so by Theorem 
14.10 there is « > 0 with the following property: for any T Cy S there is T’ Cy S 
with TNT’ = @ and |aq’| > €. Of course, if we can find such a T”, we can also find 
a T” disjoint from TUT" with |ar”| > €, and so forth: there will be a sequence 
{Tn }°2, of pairwise disjoint finite subsets of S such that for all n, |ar,| > ¢. But 
now decompose T;, = T; UT,—, where T* consists of the elements s such that 
as > 0 and T> consists of the elements s such that a, < 0. It follows that 


aT, = lar | a lar | 
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hence 

eS lar,| Slant] + laps |, 
from which it follows that max |a7+,a7,,—| > 5, 80 we may define for all n a subset 
T, C T, such that jaz, | > § and the sum ay consists either entirely of non-negative 
elements of entirely of negative elements. If we now consider TY,...,T%,,_,, then by 
the Pigeonhole Principle there must exist 1 < 71 <... < %, < 2n—1 such that all 
the terms in each TY are non-negative or all the terms in each T/ are negative. Let 
di Uja1 Tj,. Then we have a disjoint union and no cancellation, so |7;,| > 3: 
the finite sums ay are not uniformly bounded. 


PROPOSITION 14.12. Leta. : 5 > R be an S-indexed sequence with a, > 0 for 


alls € S. Then 
Sas = sup ar. 
seS TcsS 


Proor. Let A = suprc,s ar. 

We first suppose that A < oo. By definition of the supremum,, for any € > 0, 
there exists a finite subset T C S such that A—e€ < ap < A. Moreover, for any 
finite subset T’ 5 T, we have A— car < aq < A, soa. A. 

Next suppose A = oo. We must show that for any M € R, there exists a subset 
Tu Cr S such that for every finite subset TD Ty, ar > M. But the assumption 
A = oo implies there exists T Cy S such that ap > M, and then non-negativity 
gives az > ar > M for all finite subsets T” D T. 


THEOREM 14.13 (Absolute Nature of Unordered Summation). Let S be any 
set anda, : S — R be an S-indexed sequence. Let |a.| be the S-indered sequence 
s+ las|. Then a, is summable iff |a.| is summable. 


PROOF. Suppose |a.| is summable. Then for any € > 0, there exists T, such 
that for all finite subsets T of S disjoint from T., we have ||a|r| < €, and thus 


lar] =| Jas] < |S lasll = |lalr| <. 


seT seT 


Suppose |a.| is not summable. Then by Proposition 14.12, for every M > 0, there 
exists T Cy S such that |a|r > 2M. But as in the proof of Theorem 14.11, there 
must exist a subset T’ C T such that (i) az’ consists entirely of non-negative terms 
or entirely of negative terms and (ii) |jav| > M. Thus the partial sums of a, are 
not uniformly bounded, and by Theorem 14.11 a, is not summable. 


THEOREM 14.14. For ag : N > R an ordinary sequence and A € R, the 
following are equivalent: 
(i) The unordered sum D>,,¢z+ Gn 18 convergent, with sum A. 
(ii) The series \.~ , an is unconditionally convergent, with sum A. 


PROoF. (i) => (ii): Fix € > 0. Then there exists T. Cy S such that for every 
finite subset T of N containing T. we have |ar — A| < «. Put N = maxner. n. Then 
for alln > N, {0,...,n} D Te so | yp ax — Al < €. It follows that the infinite 
series ome Gy, converges to A in the usual sense. Now for any permutation o of 
N, the unordered sum Donezt @g(n) is manifestly the same as the unordered sum 
Donezt Gn, So the rearranged series Dae Gg(n) also converges to A. 

(ii) => (i): We will prove the contrapositive: suppose the unordered sum >, ex @n 
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is divergent. Then by Theorem 14.11 for every M > 0, there exists T C S with 
lar = Yer 4s| = M. Indeed, as the proof of that result shows, we can choose T 
to be disjoint from any given finite subset. We leave it to you to check that we can 
therefore build a rearrangement of the series with unbounded partial sums. 


EXERCISE 14.10. Fill in the missing details of (ii) ==> (i) in the proof of 
Theorem 14.14. 


Comparing Theorems 14.12 and 14.13 we get a second proof of the portion of the 
Main Rearrangement Theorem that says that a real series is unconditionally con- 
vergent iff it is absolutely convergent. Recall that our first proof of this depended 
on the Riemann Rearrangement Theorem, a more complicated result. 

On the other hand, if we allow ourselves to use the previously derived result 
that unconditional convergence and absolute convergence coincide, then we can get 
an easier proof of (ii) = > (i): if the series 57, an is unconditionally convergent, 
then 5°, |an| < co, so by Proposition 14.10 the unordered sequence |a.| is summa- 
ble, hence by Theorem 14.12 the unordered sequence a, is summable. 


To sum up (!), when we apply the very general definition of unordered summa- 
bility to the classical case of S = N, we recover precisely the theory of absolute 
(= unconditional) convergence. This gives us a clearer perspective on exactly what 
the usual, order-dependent notion of convergence is buying us: namely, the theory 
of conditionally convergent series. It may perhaps be disappointing that such an 
elegant theory did not gain us anything new. 

However when we try to generalize the notion of an infinite series in various 
ways, the results on unordered summability become very helpful. For instance, 
often in nature one encounters biseries 


and double series 


We may treat the first case as the unordered sum associated to the Z-indexed se- 
quence n ++ a,, and the second as the unordered sum associated to the N x N-indexed 
sequence (m,n) ++ Gm,n and we are done: there is no need to set up separate the- 
ories of convergence. Or, if we prefer, we may shoehorn these more ambitiously 
indexed series into conventional N-indexed series: this involves choosing a bijection 
b from Z (respectively N x N) to N. In both cases such bijections exist, in fact 
in great multitude: if S is any countably infinite set, then for any two bijections 
bi, b2 : S AN, bo 0 be : N > N is a permutation of N. Thus the discrepancy 
between two chosen bijections corresponds precisely to a rearrangement of the se- 
ries. By Theorem 14.13, if the unordered sequence is summable, then the choice of 
bijection 6 is immaterial, as we are getting an unconditionally convergent series. 


The theory of products of infinite series comes out especially cleanly in this un- 
ordered setting (which is not surprising, since it corresponds to the case of absolute 
convergence, where Cauchy products are easy to deal with). 
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EXERCISE 14.11. Let S; and S_ be two sets, and let ae: S; > R, be : So 5 R. 
We assume the following nontriviality condition: there exists 8s; € S, and 82 € Sg 
such that as, #0 andas, #0. We define (a,b). : S1 x Sp > R by 


(a, b)s — (a, b)(s1,80) == Dg, bso. 


a) Show that a. and b, are both summable iff (a,b). is summable. 
b) Assuming the equivalent conditions of part a) hold, show 


oa (a,b), = ( s ea] (x: b.] : 
sE€S1 x So 81 ES} s2€So 


c) When S; = Sy = N, compare this result with the theory of Cauchy products 
we have already developed. 


EXERCISE 14.12. Let S be an uncountable set, and let a, : S > R be an S- 
indered sequence. Show that if a. is summable, then {s € S | as #0} is countable. 


3. Abel’s Theorem 
3.1. Statement and Proof. 


THEOREM 14.15 (Abel’s Theorem). Let \>7° 9 dn be a convergent series. 


a) The series > ana” is uniformly convergent on [0, 1]. 
by We have lim, 4-5 gene” = oF ada: 


PRoorF. a) ({C, p. 47]) Since 77° 9 an1” = S09 an, convergence at x = 
1 is our hypothesis. By our work on power series — specifically Lemma 11.32 — 
convergence at 1 implies convergence on (—1,1), and thus aon nx" converges 
pointwise on [0,1]. Since we have convergence at xz = 1, it suffices to show uniform 
convergence on [0,1). Fix « > 0; because >, an converges, there is N € Z* such 
that | °°. @n| < «. Now we apply Abel’s Lemma (Proposition 10.2) with “a, 
sequence” ay,an41,..-, with “b, sequence” 2% ,a%+1,... (note this is positive and 
decreasing) and with M = | °°°_y, an|. The conclusion is that for any k € N, 


N+k 
| x anz”"|<aNM <a%e<e. 
n=N 
By the Cauchy Criterion (Lemma 13.1), ey Gnx” > f on [0,1). 


b) By part a), 0%.) @n2” is a uniform limit of continuous functions (indeed, of 
polynomials) on [0,1], so by Theorem 13.3, it is continuous on [0,1]. In particular 
yo One” 1s continues ate = Tr limg44--)7, Gat” = >, = 5 Gn: 


Usually “Abel’s Theorem” means part b) of the above result: Py aah Gy, = lim 2.45 Gyt™ 
But this is an immediate consequence of the uniformity of the convergence on (0, 1], 

so having this statement be part of Abel’s Theorem gives a stronger and also more 
conceptually transparent result. 


Above we followed an exercise in the text [C] of Cartan.© For comparison, here 
is a different proof of Theorem 14.15b) from a famous text of Rudin. 


5Here we are following our usual convention of allowing individual exercises to assume knowl- 
edge that we do not want to assume in the text itself. Needless to say, there is no need to attempt 
this exercise if you do not already know and care about uncountable sts. 

SHenri Cartan (1904-2008) was one of the leading mathematicians of the 20th century. 
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Proor. [R, Thm. 8.2] Since 5°, an converges, {a,} is bounded, so by Corol- 
lary 11.35 the radius of convergence of f(x) = 5°, dnz” is at least 1. Put A_; = 0; 


[oe 


for n > 0, put A, = a9 +...+@n; and put A= limyo0 An = 0,9 dn. Then 


N N N-1 
ie Anxw” = So (An — An-i)z” = (1-2) a Anz” + Anan. 
n=0 n=0 n=0 


For each fixed x € [0,1), we let N — oo to get 


f(x) = yo Anz” = (1-2) ye Ana 
n=0 n=0 


Now fix € > 0, and choose N such that n > N implies |A — A,| < ¢. Then, since 
(98) (l-a)S oa" =1 
n=0 


for all x € [0,1), we get 


f(x) — Al = |(1-2) 7 Anz” — A(L—2) $7 2”| = |(1—2) $7 (An — A)2”| 
n=0 n=0 n=0 


N fo) N 
<(1=2) 7 |An — Ala” +(S) (1-2) SD eS (1=2) 7 /An Ala” +e. 
n=0 n=0 


n=N+1 


The last quantity above approaches € as x approaches 1 from the left. Since € was 
arbitrary, this shows lim,_,,- f(a) = A. 


As you can see, Rudin’s proof uses much less than Cartan’s: rather than relying on 
Abel’s Lemma, a bit of partial summation is done on the fly. Moreover, most of the 
appeals to the theory of power series and uniform convergence are replaced by a 
clever introduction of the geometric series! Nevertheless I must say that, although 
Rudin’s argument is easy enough to follow line by line, in terms of “what’s going 
on in the proof” I find it absolutely impenetrable. 


The rest of this section is an extended exercise in “Abel’s Theorem appreciation”. 
First of all, it may help to restate the result in a form which is slightly more general 
and moreover makes more clear exactly what has been established. 


THEOREM 14.16 (Abel’s Theorem Mark II). Let f(x) = >, an(a — c)” be a 
power series with radius of convergence R > 0, hence convergent at least for all 
xcé€(c—R,c+R). 

a) Suppose that the power series converges atx =c+R. Then the function 
f:(e-—R,c+ R] > R ts continuous at « = e+ R: limy+(c¢ry- f(@) = 
f(c+R). 

b) Suppose that the power series converges at x =c—R. Then the function 
f:[e—R,c+ R) > R its continuous at « = c— R: limy(c_r)+ f(x) = 


f(c— R). 
EXERCISE 14.13. Prove Theorem 14.16. 
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EXERCISE 14.14. Consider f(x) = 4 = 2", which converges for all 
x € (-1,1). Show lim,_,_1+ f(x) exists, so f extends to a continuous function on 
[—1,1). Nevertheless f(—1) # lim,_,-1+ f(x). Why doesn’t this contradict Abel’s 


Theorem? 


3.2. An Application to the Cauchy Product. As our first application, we 
round out our treatment of Cauchy products by showing that the Cauchy product 
never “wrongly converges”. 


THEOREM 14.17. Let 3°*° 4 an be a series converging to A and Sy 5 bn be a 
series converging to B. As usual, we define c,h = 24 apbn—-p and the Cauchy 
product series pee Cn. Suppose that eae Cn converges to C. Then C = AB. 


PROOF.Put fo) = 50. Ont Ga): = S24 ae SA Ae) = eae 
By assumption, f(x), g(a) and h(x) all converge at « = 1, so by Lemma 11.32 the 
radii of convergence of S>, Gnx”, >>, bnx” and >>, ¢n2” are all at least one. Now 
all we need to do is apply Abel’s Theorem: 
C=A(1)~ lim h(x) = lim f(z)g9(z) 
x—1— 


x—1— 


= ( lim s(e)) (im a(e)) AT (1)g(1) = AB. 


x1 


3.3. Two Identities Justified By Abel’s Theorem. Here are two surpris- 
ing and beautiful identities. 


S(-1)r41 a a > 
(99) oe po ee Se ee? 


n=1 


(100) Se eee el 


The identity (100) was “known” to Leibniz (as would be logical, given that the 
convergence of both series follows from Leibniz’s Alternating Series Test), where 
the quotation marks are meant to suggest that Leibniz probably did not have an 
argument we would accept as a rigorous proof. Suppose someone shows you such 
identities, as I now have: what would you make of them? 


A good first reaction is to attempt numerical verification. Even this is not as 
easy as one might expect, because the convergence of both series is rather slow. 
(In particular, among all ways one might try to numerically compute 7, (100) is 
one of the worst I know.) Like any series which is shown to be convergent by the 
Alternating Series Test, there is a built in error estimate for the sum: if we cut off 
after N terms the error is in absolute value at most |ay+4i|. The problem with this 
is that the Nth terms of these series tend to zero quite slowly! So for instance, 

104 (-1)"+1 

Sioa = a ———— 0.6930971830599452969172323714... 

n=1 
(Using a software package I asked for the exact sum of the series, which is of course 
a rational number, but a very complicated one: it occupies more than one full screen 
on my computer. The amount of time spent to compute this rational number was 
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small but not instantaneous. In fact the reason I chose 10* is that the software man- 
aged this but had trouble with Sj95. Then I converted the fraction to a decimal. 
Of course a much better way to do this would be to convert the fractionals to deci- 
mals as we go along, but this needs to be done carefully to prevent rounding errors: 
to be serious about this sort of thing one needs to know some numerical analysis. ) 


By the Alternating Series Test, the difference between the infinite sum and the 
finite sum S91 is at most Te so we are guaranteed (roughly) four decimal places 
of accuracy. For comparison, computing log2 — e.g. by writing it as — log(4) and 


using the Taylor series for log(1 + x) — we get 
log 2 = 0.6931471805599453094172321214.... 


So indeed the identity (99) holds true at least up to four decimal places. If we 
wanted to do much more numerical verification than this, we would probably have 
to do something a little more clever. 


Similarly, we have 
104 


(a1) 
Ne, = 0.78542316..., 
arr 2n+1 
and by the Alternating Series test this approximates the infinite sum baer a 
to at least four decimal places of accuracy, whereas 
‘ = 0.7853981633974483096156608458 . .., 


which shows that (100) holds true at least up to four decimal places. 
Okay, so these identities are probably true: how do we prove them?!? 
We exploit our knowledge of power series. First, take f(x) = log(1 +). Then 
1 i [oe [oe) 
101 (a) = = = PP —1)"2". 
Qu) f@)= a5 = poy = Ls 


Integrating termwise gives 


( 1)tgrtt Cc (-1)"a2"+1 


n+l = 2 n+1 


(This was “known” to Newton and Leibniz.) So — aha! — we plug in x = 1 to get 


(102) log(1 + x) = f(x) = f(0)+ >> 


log(2) = log(1 +1) = 5° = 


But not so fast. Although you will find this explanation in many freshman calculus 
books, it is not yet justified. We were being sloppy in our above work by writing 
down power series expansions and not keeping track of the interval of convergence. 
The identity (101) holds for  € (—1,1), and that’s really the best we can do, since 
the power series on the right hand side does not converge for any other values of x. 
Integrating this term by term, we find that (102) holds for x € (—1,1). Above, in 
our excitement, we plugged in x = 1: so close, but out of bounds. Too bad! 
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But don’t despair: it’s Abel’s Theorem for the win! Indeed, because the series 
converges at x = 1 and the function log x is defined and continuous at x = 1, 


, oo SL) ea ee 
log(2) = log(1+1)= lim 1 = | AT 
eel) PRS ee ee WE ee = Oa 


EXERCISE 14.15. Establish (100) using the function f(x) = arctan. 
3.4. Abel Summability. 


Abel’s Theorem gives rise to a summability method: a way to extract numerical 
values out of certain divergent series >, an “as though they converged”. Instead 
of forming the sequence of partial sums A, = a9 +... +a, and taking the limit, 
suppose instead we look at lim,_,;- )>°-.y azz”. We say the series >, dn is Abel 
summable if this limit exists, in which case we write it as AD 5 dn, the Abel 
sum of the series. The point of this is that by Abel’s theorem, if a series baer An 
is actually convergent, say to A, then it is also Abel summable and its Abel sum is 
also equal to A. However, there are series which are divergent yet Abel summable. 


EXAMPLE 14.1. Consider the series )\~ ,(—1)”. As we saw, the partial sums 
alternate between 0 and 1 so the series does not diverge. We mentioned earlier 
that (the great) L. Euler believed that nevertheless the right number to attach to the 
series )~_)(—1)” is 4. Since the two partial limits of the sequence of partial sums 
are 0 and 1, it seems vaguely plausible to split the difference. 

Abel’s Theorem provides a much more convincing argument. The power series 
>, (-1)"2" converges for all x with |x| <1, and moreover for all such x we have 

[oe lo) 
1 1 
—1)"7" = —r)\* = = : 
Dat ) dU ) 1-(-xz) lz 


n=0 n=0 


and thus 


= 1 1 
li ~1)"2" = li ==. 
fe ie = go 


That is, the series )),,(—1)” is divergent but Abel summable, with Abel sum $. 
So Euler’s idea was better than we gave him credit for. 


EXERCISE 14.16. Suppose that )>7- 9 an is a series with a, > 0 for alln. Show 
the converse of Abel’s Theorem: if limy1- opp Qnt” = L, then yr 4 an = L. 


4. The Peano-Borel Theorem 


4.1. Statement. The Taylor series of a smooth (i.e., infinitely differentiable) 
function is surely one of the most useful concepts of basic calculus. It is only rea- 
sonable to concentrate first on “the good news” here: most naturally occurring 
functions, including all of the elementary functions one meets in precalculus math- 
ematics, are analytic: at every point c in their domain J there is a 6. > 0 such 
that the Taylor series Ty, converges to f on (c — 6¢,¢ + 6c). 


This good news builds up a strong intuition that smooth functions should have 
well-behaved Taylor series. In fact this is far from the case. To fix ideas, consider 
smooth functions f : R — R. Then there are two ways for f to fail to be analytic: 
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(i) At some c € R, Ty. converges only at 0. 
(ii) At some c € R, T;,. has positive radius of convergence, but there is no interval 
(c — b¢,€+5¢) about c on which Ty. = f. 


Both of these pathologies occur. In the second case, an example was constructed 
by Cauchy in 1823 and remains well known to this day. 


ef og #0 

0 z=0 

a) The function f :R — R is smooth. 

b) For alln EN, f™(0) =0. 

c) Thus the Taylor series expansion of f at 0 is the identically zero function. 
It has infinite radius of convergence and is equal to f only at the central 
point x = 0. 


PROPOSITION 14.18. Let f(x) = . Then: 


EXERCISE 14.17. Prove Proposition 14.18. 


That a smooth function can have a Taylor series which converges only at the central 
point — which we may certainly assume to be 0 — is more subtle. The first example 
of such a function was given by du Bois-Reymond in 1876 [dBR76], [dBR83]. 


It is no problem to construct a power series )> a,x" which converges only at 0. By 
the Cauchy-Hadamard formula we need only select a sequence with lim sup |a,,| a= 
oo, which as we have seen is implied by p = limn+0 ““** = 00, so e.g. So nla” 
works. But is this power series a Taylor series? It is if and only if we can find a 
smooth function f : R — R with ro = nl, ie., that 


Yn EN, f((0) = (n!)?. 


Thus the essential problem is to construct a smooth function f : R — R whose 
sequence of derivatives at 0 grows (at least along a subsequence) sufficiently rapidly, 
and in particular much more rapidly than n!. 

It is not so auspicious to “search in nature” for a function with such rapidly 
growing derivatives. It turns out to be simpler solve a more general problem: show 
that any real sequence whatsoever can serve as the sequence of derivatives at 0 of a 
smooth function f : R— R. This is the content of the following remarkable result. 


THEOREM 14.19. (Peano-Borel) Let {cy,}?2o be a real sequence. Then there 
is an infinitely differentiable function f : R — R such that for alln € N we have 


f (0) =Cp.- 


EXERCISE 14.18. Show that the Peano-Borel Theorem is equivalent to the state- 
ment that every power series is a Taylor series. More precisely, for anyc € R 
and any real sequence {a,}°9, show there is an infinitely differentiable function 
f:R-R such that the Taylor series expansion of f atc is pear An(a—c)”. 


EXERCISE 14.19. Prove du Bois-Reymond’s Theorem that there is an infinitely 
differentiable function f : R > R whose Taylor series expansion at 0 converges only 
atx =0. 


EXERCISE 14.20. Let {a,}°29 be a real sequence. 
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a) Show that 


S = {infinitely differentiable f : R — R| the Taylor series of f at 0 is oS Anx”} 
n=0 
is infinite. 
b) (This part is for those who know some linear algebra.) Let fo € S. Show: 
{f — fo | f € S} is a real vector space of infinite dimension. 


Theorem 14.19 is generally known as “Borel’s Theorem”. It was proved — along 
with the Heine-Borel Theorem ~ in the doctoral thesis of E. Borel [Bo95]. But ac- 
cording to a recent article of A. Besenyei [Be14] the result was in fact first proved 
by G. Peano in 1884, so we speak of the Peano-Borel Theorem. 


In fact Besenyei’s article is the source of this entire section, both the historical 
material and the account of Peano’s proof that we will now give. 


4.2. Proof. 


The following argument uses a bit of complex numbers, which are presented in 
more detail in a later chapter. 


Step 1: Let {an }?29, {bn} be real sequences with b, > 0 for alln € N. Put 


Suppose f converges on all of R and that for all n € N and x € R we have 
0 k \ (@) 
(")() = ue 
ro)= 3 (pea) 


Later we will show how this can be achieved with suitable choices of the sequences 
{an} and {by}. Now we have 


k lo e) co 
2 Apex _ k ind 2G jd 2 25+k 
|b,x"] <1 = eae ARE (-1) bya? = Di(- Di andj Cam: 
j=0 j=0 
Thus 
anak (n) jp n!(—1)5an—250),_94 if k =n — 29 for some j 
1+ bpx? 0 otherwise. 
Thus 
(103) f(0) = a0, f"(0) = a1, and 
(n) (9 L$] 
(104) Vn > 2: f = ) = An 4 ( 1)? an—2jb),_9;- 
! a 


From (103) and (104) it follows that given {b,} and {c,}, there is a uniquely 
determined sequence {a,,} such that f((0) = cp for alln € N. 
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Step 2: Let b > 0 and consider 


ak > 1 1 
be+a72 °° 2 xetbi x—bi)’ 


By the Generalized Leibniz Rule, we have 


ak (n) 1 n - | oo 
(gis) ~ 5 (") =) b= 1 nt ae! 1l-n+j 
j=0 


( (Deg. eo ee ) 
(x + bi)i#1 " (2 — bi) 


_ 7 kn-2 “ gle Dah Eaner 9s) meee | ait! 
2 2 ( (; | ) 


n—j)! xt bijitt — (x — bi)si+t 


We have (see the following exercise) 


gi tt gi tt 


(nee) (c+ bist " (x — bijs+l 


< 2. 


It follows that 


k (n) 

px (n+ 1)IK lan], p-n—2 
106 Vk > 2 — < tA Wee 
(106) >n42, |( HE) |< Ce 
Taking by = (k!)?|a,|, we get 

k \(™) k—n-2 

Apx |2| 
1 —— < 1)! a 
(107) Ss (5) S(n+D! Do ay 
k>n+2 k>n+2 


The right hand side of (107) is uniformly convergent on every bounded interval, 
hence for every n € N the convergence of 
= ( apa a 
2 
a 1+ bpx 
follows from the Weierstrass M-Test. Applying Theorem 13.6 establishes the fact 


that the given series of nth derivatives converges to the nth derivative of f. This 
completes the proof. 


EXERCISE 14.21. a) Letw € C. Show that if |w| <1, then for alln € 
Zt, |\w" +w"| < 2. 
b) Verify equation (105). 


5. The Weierstrass Approximation Theorem 
5.1. Statement of Weierstrass Approximation. 


THEOREM 14.20 (Weierstrass Approximation Theorem). Let f : [a,b] + R be 
a continuous function and € any positive number. Then there exists a polynomial 
function P such that for all x € [a,b], |f(z) — P(x)| < €. In other words, any 
continuous function defined on a closed, bounded interval is the uniform limit of a 
sequence of polynomials. 
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EXERCISE 14.22. For each n € Zt, let P, :R — R be a polynomial function. 
Suppose that there is f : R > R such that P, + f on all of R. Show that the 
sequence of functions {P,,} is eventually constant: there exists N € Z* such that 
for allm,n>N, Pr(x) = P(x) for alla eR. 


It is interesting to compare Theorem 14.20 with Taylor’s theorem, which gives 
conditions for a function to be equal to its Taylor series. Note that any such 
function must be C® (i.e., it must have derivatives of all orders), whereas in the 
Weierstrass Approximation Theorem we can get any continuous function. An im- 
portant difference is that the Taylor polynomials T(x) have the property that 
Tn41(2) = Ty (x) +ay412%, so that in passing from one Taylor polynomial to the 
next, we are not changing any of the coefficients from 0 to N but only adding a 
higher order term. In contrast, for the sequence of polynomials P,,(”) uniformly 
converging to f in Theorem 1, P,+41(«) is not required to have any simple algebraic 
relationship to P,,(x). 


Theorem 14.20 was first established by Weierstrass in 1885. To this day it is one 
of the most central and celebrated results of mathematical analysis. Many mathe- 
maticians have contributed novel proofs and generalizations, notably S.J. Bernstein 
[Be12] and M.H. Stone [St37], [St48]. But — more than any result of undergrad- 
uate mathematics I can think of except the quadratic reciprocity law — the 
passage of time and the advancement of mathematical thought have failed to single 
out any one preferred proof. We have decided to follow an argument given by Noam 
Elkies.” This argument is reasonably short and reasonably elementary, although as 
above, not definitively more so than certain other proofs. However it unfolds in a 
logical way, and every step is of some intrinsic interest. Best of all, at a key stage 
we get to apply our knowledge of Newton’s binomial series! 


5.2. Piecewise Linear Approximation. 


A function f : [a,b] + R is piecewise linear if it is a continuous function made up 
out of finitely many straight line segments. More formally, there exists a partition 
P={a=2 < 2... < £, = b} such that for 1 <i < n, the restriction of f to 
[x;-1, v;] is a linear function. For instance, the absolute value function is piecewise 
linear. In fact, the general piecewise can be expressed in terms of absolute values 
of linear functions, as follows. 


LEMMA 14.21. Let f : [a,b] > R be a piecewise linear function. Then there is 
n€Z* and aj,...,4n,™1,...,M%n,0 € R such that 


f(x) =b+ 5 t}mia; + bi. 
i=1 


PROOF. We leave this as an elementary exercise. Some hints: 
(i) If a; <a, then as functions on [a,b], |v — a;| = x — aj. 
(ii) The following identities may be useful: 


“http: //www.math.harvard.edu/~elkies/M55b.10/index.html 
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_lftagl_ lf-al 
2 a 
(iii) One may, for instance, go by induction on the number of “corners” of f. 


min(f,g) 


Now every continuous function f : [a,b] > R may be uniformly approximated 
by piecewise linear functions, and moreover this is very easy to prove. 


PROPOSITION 14.22 (Piecewise Linear Approximation). Let f : [a,b] > R be a 
continuous function and € > 0 be any positive number. Then there exists a piecewise 
linear function P such that for all x € [a, 6], |f(z) — P(x)| <e. 


PRooF. Step 1: Let f : [c,d] — R be a continuous function, and put M = 
w(f, [c, d]) = max(f, [c,d]) — min(f, [c, d]). Let L : [c,d] — R be the unique linear 
function with L(c) = f(c) and L(d) = f(d). We cLaim that for all x € [c,d], 
| f(a) — L(x)| <M. To establish the CLAIM we may argue (for instance) as follows: 
for all x, either f(x) > L(x) or f(x) < L(x). In the former case, 


Lf a) — L(2)] < max f, c, dl) — min(L,[¢, dl) = max(f,[e, dl) — min(L(c), E(a)) 
= max(f, [e, d]) ~ min(f(c), f(d) < max(f, [c, dj) = min(f, le d]) = M, 


and the argument in the latter case is very similar. 
Step 2: By the Uniform Continuity Theorem, there is 56 > 0 such that whenever 
|x —y| < 6, | f(x) — f(y)| < €. Choose n large enough so that =* < 6, and consider 
the partition P, = {a = 2% <4 <...<%n_1 < &y} of [a,b] into n subintervals of 
equal length 2“. Let P : [a,b] + R be piecewise linear, linear on [x;, 7;4] and such 
that P(x;) = f(a;) for allO <i <n. Then for all0 <i<n, w(f,[%:,2i41]) < €, so 
by Step 1 | f(x) — P(«)| < ¢ for all x € [x;, 2:4] and thus for all x € [a, 8). 


5.3. A Very Special Case. 
LEMMA 14.23 (Elkies). Let f(z) = >7° 5 @n2” be a power series. We suppose: 


(i) The sequence of signs of the coefficients an is eventually constant. 
(ii) The radius of convergence is 1. 
(iii) The limit lim, _,,- f(a) = L evists. 
Then So) @n = L, and the convergence of the series to the limit function is 
uniform on [0, 1]. 


EXERCISE 14.23. Prove Lemma 14.23. Two suggestions: 


(i) Reduce to the case in which an > 0 for alln EN. 
(ii) Use the Weierstrass M- Test. 


PROPOSITION 14.24. For any a > 0, the function f(x) = |x| on [—a,a] can be 
uniformly approximated by polynomials. 


PROOF. Step 1: Suppose that for all « > 0, there is a polynomial function 
P:[-1,1] > R such that |P(«) — |a|| < ¢ for all x € [-1,1]. Put « = 4. Then for 
all y € [—a, a] we have 


laP(2) — a|2 || = |Q(y) — lull < ae, 


where Q(y) = aP(*) is a polynomial function of y. Since € > 0 was arbitrary: if 
x +> |x| can be uniformly approximated by polynomials on [—1, 1], then it can be 
uniformly approximated by polynomials on [—a, a]. So we are reduced to a = 1. 
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Step 2: Let Ty (y) be the degree N Taylor polynomial at zero for the function 
f(y) = V1—y. By the Binomial Theorem, 


(l+y)* = = (“)n 


valid for all a € R and all y € (—1,1). Taking a = $ and substituting —y for y, we 
see that the degree N Taylor polynomial for /1— y at zero is 


WN 1 
—_ _ n 2 n 
Tv) = ep" (2)y" 
and limy-..0Tw(y) = /1— y for y € [0,1). Further, (-1)"(2) <0 forn>1, and 


lim f(y)= lim /1-y=0. 
yori 


yo l- 


Thus we may apply Elkies’ Lemma to get Ty (y) + I — y on (0, 1]. For x € [—1, 1], 
y =1-—2? € [0,1], so making this substitution we find that on [—1, 1], 


N 1 
T(t - 2%) = )o(-1"(2)a a7)” & 4/1 — (1-2?) = v2? = (a. 


n 


n=0 


5.4. Proof of the Weierstrass Approximation Theorem. 


For a < b € R, let C[a,b] be the set of all continuous functions f : [a,b] > R, 
and let P be the set of all polynomial functions f : [a,b] + R. Let PL({a, b]) denote 
the set of piecewise linear functions f : [a, b] > R. 


For a subset S C C[a,b], we define the uniform closure of S to be the set S 
of all f € C[a,b] which are uniform limits of sequences in S: precisely, for which 
there is a sequence of functions fy, : [a,b] > R with each fn € S and fn + f. 


LEMMA 14.25. For any subset S C C[a, b], we have S=S. 


PROOF. Simply unpacking the notation is at least half of the battle here. Let 
f €S, so that there is a sequence of functions g; € S with g;  f. Similarly, since 
each g; € S, there is a sequence of continuous functions fij 4 gj. Fix k € Zt: 
choose n such that ||gn — f|| < s¢ and then j such that ||fnj — gn|| < s;3 then 


1 1 1 
ceo FS lifes + ||9n OR 
Ilfng — fll S Il fag — nll + Ilgn — fll < ap + og = | 


Thus if we put f, = fnj, then for all k € Z*, || fx — f || < ¢ and thus f;, 4 f. 


Observe that the Piecewise Linear Approximation Theorem is 


PL[a, b] = C[a, 6], 


whereas the Weierstrass Approximation Theorem is 


P= Cla; 0). 
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Finally, the point: it’s enough to show that every piecewise linear function can be 
uniformly approximated by polynomial functions, for then P > PL|{a, }], so 


P =P > PLfa,}] =C[a, d. 
Thus the following result completes the proof of Theorem 14.20. 
PROPOSITION 14.26. We have P D PL{a, 8. 


ProoF. Let f € PL[a,b]. By Lemma 14.21, we may write 


f(a) =b+ dot lmie + bi]. 


Choose a > 0 such that for all 1 <i <n, if x € [a,b], then m;x +b; € [-a,a]. For 
each 1 <i <n, by Lemma 14.24 there is a polynomial P; such that for all x € [a,b], 
|Pi(mx+b;)—|mz+b;|| < £. Let P : [a,b] 3 R by P(x) = b+ 77, +P;(mixtbj). 
Then P € P and for all x € [a, 8], 


n n 


|P(a) — f(a)| = |S> £(Pi(mia + b;) — |mix + bi) < > - = 


i=l i=1 


6. A Continuous, Nowhere Differentiable Function 


We are going construct a function f : R > R with the following striking property: 
for all zo € R, f is continuous at 29 but f is not differentiable at xo. In short, we 
say that f is continuous but nowhere differentiable. 


The first such construction (accompanied by a complete, correct proof) was given 
in a seminal 1872 paper of Weierstrass. Weierstrass’s example was as follows: let 
a € (0,1), and let b be a positive odd integer such that ab > 1+ 32. Then the 
function f : R > R given by 


(108) f(x) = 2S a” cos(b" rx) 
n=0 


is continuous on R but not differentiable at any x € R. 
EXERCISE 14.24. Show that the function defined by (108) above is continuous. 


Unfortunately the proof that f is nowhere differentiable is not so easy, as indicated 
by the rather specific conditions given on the parameters a,b. (For less carefully 
chosen a,b the function f can have a “small” set of points of differentiability.) 
Thus, as with most other contemporary treatments, we will switch to a different 
function for which the nowhere differentiability calculation is more straightforward. 
More specifically, we will switch from trigonometric functions to our new friends 
the piecewise linear functions, so first we interpose the following exercise nailing 
down some further (simple) properties of these functions. 


EXERCISE 14.25. a) Let f : [a,b] > R be a piecewise linear function with 
slopes m1,...,Mn. Show that f is Lipschitz, and the smallest possible 
Lipschitz constant is C = max; |m;|. 
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b) Let f : R > R be a function. Suppose that there is C > 0 such that 
for every closed subinterval [a,b] of R, C is a Lipschitz constant for the 
restriction of f to [a,b]. Show that C is a Lipschitz constant for f. 

c) Let f : R > R be a piecewise linear function with “corners” at the in- 
tegers — t.e., f is differentiable on (n,n +1) for all n € Z* and is not 
differentiable at any integer n. For n € Z, let my be the slope of f on 
the interval (n,n+1). Let C = supyezmn. Show that f is Lipschitz iff 
C < oo, in which case C is the smallest Lipschitz contant for f. 


Now we begin our construction with the “sawtooth function” S : R — R: the unique 
piecewise linear function with corners at the integers and such that S(n) = 0 for 
every even integer n and S(n) = 1 for every odd integer n. The slopes of S are all 
+1, so by the preceding exercise S is Lipschitz (hence continuous): 

Va,y €R,|S(x) — S(y)| < |x — y}. 
Also S$ is 2-periodic: for all zc € R, S(x+2) = S(az). For k EN, define 


fe: RR, fk(2) = (3) S(4* 2). 


We suggest that the reader sketch the graphs of the functions f,: roughly speaking 
they are sawtooth functions which, as & increases, oscillate more and more rapidly 


but with smaller amplitude: indeed || fx(x)|| = (33°, We define f : R > R by 


co fee) k 
i@=s 1@=s- (3) S(4*e). 
k=0 


k=0 


oice 2p ll Fell = Se ay < oo, the series defining f converges uniformly by 
the Weierstrass M-Test. This also gives that f is continuous, since f is a uniform 
limit of a sequence of continuous functions. We claim however that f is nowhere 
differentiable. To see this, fix zo € R. We will define a sequence {6,,} of nonzero 
real numbers such that 6, — 0 and the sequence 


f(@0 + 4n) — fo) 
dn 
is divergent. This implies that f is not differentiable at xo. 
1 


Let’s do it. First suppose that the fractional part of xo lies in (0,5), so that 


D,, = 


the interval (x, xo + 4) contains no integers. In this case we put 

4nr 

“9? 

and the reason for our choice is that the interval (4"29,4"(ao9 + 6,)) contains no 
integers. Let k,n € N. We claim the following inequalities: 


bn = 


(109) Vk >n, |S(4*a0 + 4*5,) — S(4*a0)| = 0. 
(110) Vk =n, |S(4*aq + 4*5,) — S(4*ax9)] = - 
(111) ]vk <n, |S(4¥ 2 + 4*6,) — S(4*20)| < |4* 6]. 


Indeed: (109) holds because if k > n, 4*a9+4"5, —4* a = 4"6, = aes is a multiple 


of 2 and S is a 2-periodic function; (110) holds because if k = n, 4" a9 + 4°56, = 
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AFag + 4, so by our choice of 6,,, the function S is linear on [4* xo, AFag + 3] of slope 


+1, hence the difference between its values at the endpoints is a Finally, (111) 
holds because 1 is a Lipschitz constant for S. Using these results and the Reverse 
Triangle Inequality gives 


a + bn) — f (x0) 


(3) * g(4*29 + 4*5,) — S(4* a0) 


On 4 On 
k=0 
n n-1 k 
3 3 (4% a9 + 4*5,) — S(4® 20) 
> 4” 5 
k=0 
date 3-1 3" 
> gn k n as 
3 a 3 5 : 


Thus D, — c, so f is not differentiable at xo. 
We're not quite done: recall that we assumed that the fractional part of xo lay 


in [0, ), with the consequence that S was linear on the interval [(4"x9, 4" (xo +5n)]- 


2 
What to do if the fractional part of zo lies in [5, 1)? In this case we take 5, = — 
so that the interval (4"(zo + 6,),4"x%0) contains no integers so S is linear on the 
interval [4"(2o + 6,),4" 20], and the rest of the proof goes through as above. 


So, albeit with a different function, we have proved Weierstrass’s Theorem. 


THEOREM 14.27. (Weierstrass, 1872) There is a function f : RR — R which is 
continuous at every point of R but differentiable at no point of R. 


Notice that if we restrict f to some closed interval, say [0, 2], then by the Weierstrass 
Approximation Theorem f is — like any continuous function on [0,2] — a uniform 
limit of polynomials. Thus even a uniform limit of polynomials on a closed, bounded 
interval need not have any good differentiability properties whatsoever! 


7. The Gamma Function 


There are many real functions. Some are “elementary”: i.e., built up from power 
functions, trigonometric functions, exponentials and logarithms. Most are not, like 
the Gaussian error function E(x) = fee e~* dt. As mathematical analysis devel- 
oped from Newton and Leibniz in the 17th century on down, eventually it became 
clear that certain non-elementary functions arise again and again in a variety of 
contexts, and satisfy some remarkable, beautiful identities. 

Such functions are called special functions. There is no precise mathematical 
definition of a special function. Rather, the appellation is historical and cultural. 
Unfortunately the days when the majority of students and practitioners of math- 
ematics naturally learned about special functions in the context of their work are 
past, since indeed it is no longer the case that that the majority of students and 
practitioners of mathematics are deeply concerned with real function theory. In 
our day your smart friend the physicist or the engineer probably knows more about 
special functions than you do...and this may not be a good thing. 

In this section we will study one of the most ubiquitous special functions of 
them all, the Gamma function. 
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7.1. Definition and Basic Properties. 


For x € (0,00) we define 
I(a2) = i‘ t?-1e-'dt. 
As this is an improper integral, thre is ‘of course something to check. 
PROPOSITION 14.28. For x € (0,00), the integral defining T(x) is convergent. 


EXERCISE 14.26. Prove Proposition 14.28. Some suggestions: 

(i) First deal with the case x >1, since in this case the function t t*~1 is 
continuous on [0, 00) and the integral is “improper only at co. To show the 
convergence use the fact that exponentials grow faster than polynomials. 

(ii) When 0 < x <1, the integral is also improper at 0, so it should be split 


into two pieces, say ahs and Aas The argument of part a) will handle the 


dx 


latter integral. Reduce the former integral to ie re 


EXERCISE 14.27. a) Show that the improper integral ie t®—-1e-'dt is di- 
vergent for x <0. 
b) Show that I(x) > 0 for all x € (0,00). 
c) Show that lim,_,9+ T(x) limy_,.. I'(x) = ov. 
d) Show that [(1) =1(2) =1. 


EXERCISE 14.28. a) By making the change of variables t = s”, show that 
for all x € (0,00), 


(112) I(2) = 2 | g2?—le-*" de, 
0 
b) Deduce that 


1 co co 
(113) r (5) = 2 | e* dr = i ede. 
2 0) —oo 


THEOREM 14.29. a) For all x € (0,00) we have 


(114) I(a+1) =aI (a). 
b) For alln € N, T(n4+1) =n. 
EXERCISE 14.29. a) Prove Theorem 14.29a). 


(Suggestion: integrate by parts.) 
b) Prove Theorem 14.29b). 


Thus the Gamma function is a continuous interpolation of the factorial function 
(with a slight shift in the argument that everyone finds a little distressing at first 
but eventually learns to live with). Of course there are infinitely many continuous 
(even infinitely differentiable) functions which interpolate any real sequence. Is 
there a sense in which the Gamma function is the “right” interpolation? 

Yes, although it is rather curious. 


THEOREM 14.30 (Bohr-Mollerup [BM22]). a) The function logT : (0,00) > 
R is convex. 
b) Conversely, let f : (0,00) + (0,00) satisfy: 
(i) f(w@ +1) =af(x) for all x € (0,0); 
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(ii) f(1) = 1; and 
(iii) log f is convex. 
Then f =T. 


PROooF. a) Let f = log’. We will verify the secant-graph inequality: for all 
0<a<y<oanddA€ (0,1), 


f(A = Aja + Ay) S (1 — ANF(@) + AF(Y); 
and for this we will use Hélder’s Integral Inequality with p = t and q= —: 


f(a = A)a 4 Xb) = log | pPrt(—Ajy-le-t ay 
0 


= log (| (ete te at) 
0 
oo » oo 1-r 
< log (/ teat) (/ treat) 
0 0 


= Alog[(x) + (1 — A) log l'(y). 
b) In view of part a), it suffices to show that there is a unique function satisfying 
conditions (i), (ii) and (iii). Because conditions (i) and (ii) define f on (1,00) in 
terms of its values on (0,1), it is enough to assume that x € (0,1). 
Put 


yp = log(f): (0,00) ~R 
so y(1) = 0, ¢ is convex and 
(115) Va € (0,00), p(a+1) = v(x) + log z. 
Applying induction to (115) we get 
Vn € Zt, p(n +1) = log(n!). 


Let « > 0. Applying the three-secant inequality to y withh<n+1<n+1+2 
we get 


p(n +1) — (rn) — p(n+1+2) -v(n+)) 
(n+1)-n ~ (n+14+a2)-—(n4+1) ’ 


which resolves to 


logn < Ltt) — y(n +1) 
xv 


Applying the three-secant inequality to y withn+1<n+1+a<n-+2 gives 


p(nt+1+2)—y(n4+1) | y(n+2)—-y¢(n4+1) 
= | 1 
i Gada 
so overall we get 
ead ! 
logn < Piet Lm) eee) < log(n + 1). 


x 
Repeated application of (115) gives 
olnt 142) = ole) + log (o(e+1)--- (e+), 
so 
(0) + log (ali +1)---(e-+n)) < pnt 142) > alogn + o(n+ I), 
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and rearranging gives 


x 


x(a + re + >) , 


p(x) > xlogn + log(n!) — log(a(a + 1)--- (a +n)) = log ( 


Similarly we have 


ple) +log(e(w + 1)---(@+n)) = y(n +142) <elog(n + 1) + log(n!), 


p(x)—log Ge ie) < rlog(n+1)+log(n!)—log(n!) —log(n*) = rlog(1++), 


so overall we get 


0 < y(x) —log i ) < log (1+ *) 


Taking the limit as n — oo shows that 


ie) = ee (=x ae 5G + =) ; 


From the end of the proof of Theorem 14.30 we get that for all x € (0, 1), 
n*n! 

116 T(x) = li ‘ 
ee oe commas) 

EXERCISE 14.30. Use (114) and (116) to show that 

n*n! 

11 I(x) = li ‘ 
(117) Va € (0,00), I(x) im (aes) 


It seems interesting to investigate non-integer values of the Gamma function. In 
some sense the first order of business is to evaluate I'(4). Then using (114) we 


can evaluate ['(#) for all integers n. Moreover, by (113) we know that I'($) = 


y Sateen er dx, an improper integral that is easily seen to be convergent but whose 
precise value is not so easy to determine. 


To evaluate I'($) we introduce a second special function and relate it back to the 
Gamma function using the Bohr-Mollerup Theorem. 


7.2. The Beta Function. 


For x,y € (0,00), we define the Beta function 


B(a,y) = | t7-1(1 — t)9 "dt. 


When z,y > 1, this is a “proper integral”, and there are no convergence issues. 
However, when one of x, y is less than 1, there is something to check. 


EXERCISE 14.31. Show that the integral defining B(x,y) is convergent for all 
x,y >0. 


THEOREM 14.31. For all x,y > 0, we have 


Pw)P(y) 


B(x, y) = Deal 
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EXERCISE 14.32. This exercise develops a proof of Theorem 14.31. 
a) Show: for all y > 0 we have B(1,y) = = 
b) Show that for each fixed y € (0,00), the function log B(x, y) is a conver 
function of x. 
(Suggestion: adapt the proof of Theorem 14.30a).) 
c) Use integration by parts to show that 


x 
118 Ve,y >0, Bia+1,y) = ——B(a,y). 
(118) x,y Phy) = Pee) 
d) For each fixed y € (0,00), apply the Bohr-Mollerup Theorem to 


T(¢+y) 
f(z) = =~ Bla, y) 
ae 
to deduce that f(x) =I(z). 
Making the substitution t = sin? @ in the integral defining B (x,y) and applying 
Theorem 14.31 we get 


Pins Soo : D(2)P(y) 
119 2 | sin 0)**~* (cos)*4~! d0 = . 
(119) 3 ee Kee) T(x +y) 
Taking « = y = 4 in (119) we deduce: 


THEOREM 14.32. 
1 lee) 
(120) 8 (5) =f cdr = yr. 
EXERCISE 14.33. Show: for alln € Z*, we have 
1 (2n!) fr 
E (n+ 5) ~ Aryl * 
7.3. Interlude: A Dominated Convergence Theorem. 


THEOREM 14.33. Let fn, f,g : (0,00) > R. We suppose: 
(i) fr and g are Riemann integrable on every closed, bounded subinterval of 
(0, 00); 
(ii) fn > f pointwise, and the convergence is uniform on each closed, bounded 
subinterval of [0, 00). 
(iii) |fn(x)| < g(x) for all x € (0,00); and 
(iv) f° g(@)dax < 00. 
Then limy-+o0 i fr(a)dx = ee f(a)dax. 


PRooF. Since |fp(x)| < g(a) for all x and f, — f, also |f(x)| < g(x) for all a. 
Fix € > 0. Since g is non-negative and ff g(a)dx < 00, there is N € Z* with 


ie aod: ie dei dees, 


N 
Thus for all n € Zt, 


i tied y - inla)ide +f” [ialo)lde <e 


+f? stent 
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and similarly 


[ * sa)ae 


Since f,, > f uniformly on [ 


< | * pa)lde + [ e@lar <e 


N 


4 ie f(a)dx 
N}, 


i 
N? 


n—-oo 


N N 
lim fn(ax)dxz = f(a)dz. 


It follows that for all sufficiently large n, 


[ fr(x)dx =f f(a)dex 


< (ee Mfn(a)|da + (ae If(2)|de + 


e+ete=de. 
Since € was arbitrary, the proof is complete. 


N 
1 


a (fala) — fla) de 


Remark: Theorem 14.33 is given as an exercise in W. Rudin’s text [R, p. 167]. 
It is a weak form of what is perhaps the most important and useful single result 
of graduate level real variable theory, the Lebesgue Dominated Convergence 
Theorem. In Lebesgue’s version, the integrals are taken in his more permissive 
sense (which we certainly have not discussed). More significantly, the hypothesis of 
uniform convergence on closed bounded subintervals is dropped entirely: all that is 
needed is that f, — f pointwise. 

Since uniform convergence on bounded subintervals is a much stronger hypoth- 
esis than pointwise convergence, Rudin’s version of the Dominated Convergence 
Theorem is significantly weaker than Lebesgue’s. (For a version of the Dominated 
Convergence Theorem which uses only pointwise convergence but stays in the con- 
text of the Riemann integral, see [Ke70]. But I must warn you that even this paper 
requires somewhat more background than we have developed here.) Nevertheless 
Theorem 14.33 has some useful applications. 


7.4. Stirling’s Formula. 


THEOREM 14.34. 


. T(i@+1) 
(121) im (a/e)* Jona =1 


PROOF. ... 


CHAPTER 15 


Several Real Variables and Complex Numbers 


1. A Crash Course in the Honors Calculus of Several Variables 


Let M,N € Z+. In multivariable calculus one studies functions f : RY > R™, 
that is “vector valued functions of several variables”. Here we will be briefly inter- 
ested in several aspects of the case M = 1, ie., of functions f : RN — R and also 
of sequences and series with values in RY. 


First, recall that by RY we mean the set of all ordered N-tuples « = (%1,..-,0Nn) 
of real numbers. To define things like continuity and convergence, need a way to 
measure the distance between x,y € R%. For this, we define 


le] = [(er,...,2n)| = fae +... +23. 


Notice that, at least when 1 < N < 3, this is the distance from the origin 0 = 
(0,...,0) to the vector x that we learn about in high school geometry. The basic 
fact that allows estimates to be made is the triangle inequality: 


Va,y ERY, |x +y| < lel +Iy- 
We proved this at the end of Chapter 8 as an application of Jensen’s Inequality. 


Here are the two key definitions: a sequence {x,}9@, in R% is given by a function 
nt> an from Zt to RN. We say that the sequence x, converges to x € R% if for 
all « > 0, there exists No € Z* such that for all n > No, |an — | < €. A sequence 
is convergent if it converges to some element of R™ and otherwise divergent. 


Let D C RY, and let x € D. A function f : D > R is continuous at x if for all 
€ > 0, there exists 6 > 0 such that for all y € D with |x — y| < 6, |f(xz)—f(y)| <e. 
A function f : D > R is continuous if it is continuous at every point of D. 


EXERCISE 15.1. Let « = (%1,...,@N),y = (Y1,---,yn) € R®. Show that for 
alll <i<N, |x; —y| < |x — yl. 


EXAMPLE 15.1. For 1 <i< N, let a: RN > R be the function which maps 
(a1,...,@n) to aj. Then 7; is continuous: indeed, for any « > 0, we may take 
6 =e, since then if |x — y| < 6, |m(x) — mily)| = |e — yi| < |2 — y| <€. 


EXERCISE 15.2. Fiz 29 € R%, and consider the function dz. ? RY > R, 
dz (x) = |x — xo|. Show that dz, is continuous. 


PROPOSITION 15.1. Let fi, fo: DCR > R be functions, and let a, 6 €R. 


a) Tf fi and fz are continuous, so is af; + Bfo. 
b) Jf fi and f are continuous, so is fy fo. 


323 


324 15. SEVERAL REAL VARIABLES AND COMPLEX NUMBERS 


c) If fi and fz are continuous and for all x € D, fo(x) 4 0, then £ is 
continuous. 


EXERCISE 15.3. Prove Proposition 15.1. (Suggestion: adapt the N = 1 case.) 


EXERCISE 15.4. a) Give a careful definition of a polynomial function f : 
RY SR. 
b) Show that every polynomial function is continuous. 


PROPOSITION 15.2. Let {an} be a sequence in RN, and let x € RN. TFAE: 
(it) For alll <i<n, (an) > 73(2). 


PRooF. (i) ==> (ii): Fix e« > 0. By assumption, there is No € Zt such 
that for all n > No, |an —2| < €. Hence by Exercise 15.1, for all 1 <i < N, 
\7i(tn) — mi (2)| <€. 

(ii) => (i): Fix « > 0. For each 1 < i < N, there exists N; € Z* such that 
for all n > Ni, |mi(an) — m(x)| < €. Take No = max(M,...,Nw); for n > No, 
|tn—2| = J (nt 1)? +...4+(fn,n — tN)? < VNé = VNe. Good enough. 


PROPOSITION 15.3. Let f : DC RN — R be a continuous function, and let 
{tn} be a sequence in D. if tn > x, then f(an) > f(a). 


EXERCISE 15.5. Prove Proposition 15.3. 


We now press on to give analogues of the fundamental Interval Theorems of 
honors calculus. Our first order of business is to find a suitable analogue in R“ 
of the notion of a closed, bounded interval in R. It turns out that we have some 
leeway here. Let us consider two different kinds of sets. 


For 1 < i < N, choose real numbers a; < b;. We define the closed coordi- 
nate box [a,b] to be the set of all x = (a1,...,an) € R™ such that a; < 2; < b; 
for all 1 <i< N. Further, for x € R% and r > 0, we define the open ball B,(x) 
to be the set of all y € R% such that |x — y| < r and the closed ball B,(x) to be 
the set of all y € R% such that |x — y| <r. 


EXERCISE 15.6. For a subset D of R', show that the following are equivalent: 
(i) There is some closed coordinate box [a,b] with D C [a,b]. 
(ii) There arex € RN andr >0 with DC B,(z). 
(iii) There is r > 0 with D C Bo(2). 
A subset D satisfying these equivalent properties is bounded. 


EXERCISE 15.7. Let D C R™ be a bounded subset. For all 1 <i< N, show 
thagt m;(D) is a bounded subset of R. 


EXERCISE 15.8. Show that all closed coordinate boxes and all open and closed 
balls are convex subsets of RN. 


THEOREM 15.4. Let DC R™ be a nonempty convex subset, and let f : D +R 
be a continuous function. Then f(D) = {f(a) | « € D} is an interval in R. 


ProoFr. By Theorem 8.1, it is enough to show f(D) is convex, i.e., if y1,y2 € 
f(D), then the entire interval from yi to y2 is in f(D). Suppose y: = f(a1) and 
yo = f(a). Define a function g: [0,1] > R by g: th f((1—t)a, + tr2). The 
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function g is well-defined since D is convex; it is continuous since f is continuous, 
and g({0,1]) c f(D). By the usual Intermediate Value Theorem, g((0,1]) is an 
interval in R. Since g(0) = f(#1) = yr: and g(1) = f(x2) = ye, this implies that 
every point between yi and yp lies in f(D). 


THEOREM 15.5 (Multivariate Bolzano-Weierstrass Theorem). Every bounded 
sequence in RN admits a convergent subsequence. 


ProoFr. When N = 1 this is the usual Bolzano-Weierstrass Theorem. We will 
reduce to this case by repeated passage to subsequences. Let {x,,} be a bounded 
sequence in R". By Exercise 15.8, 71 (an) is a bounded sequence of real numbers. 
By the usual Bolzano-Weierstrass Theorem, there is a subsequence nz, such that 
T1(%n,) converges. Since {z,,} is bounded, so is its subequence z,,, and hence 
so is 72(an,). Applying Bolzano-Weierstrass again, we get a subsubsequence Ln, 


such that T2(Xn,,) converges. Continuing in this manner — i.e., passing again to a 
subsequence for each coordinate in turn — we eventually get a subsub...subsequence 
(N “subs” in all!) all of whose coordinates converge. This is a subsequence of the 
original sequence which converges by Proposition 15.2. 


A subset D C RN is closed if for all sequences {xn} with values in D, if rv, > x € 
R then z € D. 


EXERCISE 15.9. Show that a closed interval I C R is a closed subset of R. 


PROPOSITION 15.6. Closed coordinate boxes [a,b] and closed balls B,(x) are 
closed subsets of RN. 


Proor. Let D = [a,b] = TI, as, b;] be a closed coordinate box, and let z,, be 


a sequence in D converging to  € RN. Then for 1 <i < N, 7;(an) > 7i(x). Since 
Ti(@n) € [az, bi], 7i(x) € [ai, b;]. It follows that x € [a,b] and thus [a, }] is closed. 
Let D = B,(xq) be a closed ball and let 2, be a sequence in D converging to 
a € RY. By Exercise 15.3, y + |ao —y| is continuous, and thus |%9 —2n| > |zo —2I. 
Since |xo — @n| <r for all n, |zo —2| <r, ie., ¢ € B,(xo): B,(xo) is closed. 


THEOREM 15.7 (Multivariable Extreme Value and Uniform Continuity). Let 
D CRN be closed and bounded. 


a) Every sequence in D admits a subsequence converging to an element of D. 

b) Every continuous function f : D + R is bounded and attains its maximum 
and minimum values. 

c) Every continuous function f : D— R is uniformly continuous. 


PROOF. a) Since D is bounded, by Theorem 15.5, every sequence in D admits 
a subsequence converging to « € R‘. Since D is closed, 2 must lie in D. 
b),c) The proofs of Theorems 10.20 and 10.21 which treat the case D = [a, b] adapt 
easily to the general case. Details are left to the reader. 


2. Complex Numbers and Complex Series 


It turns out that the sine, cosine and exponential functions are all very closely 
related to each other, provided we are willing to work with complex values. In 
this section we provide a review of complex numbers and the rudiments of complex 
power series. This theory can be developed to an amazing extent — further than the 
theory of real power series, in fact! — but such qualitatively different developments 
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are the subject of another course. Here we just want to develop the theory enough 
so that we can make sense of plugging complex numbers into power series. 


Recall that a complex number is an expression of the form z = a+ bi. Here a 
and b are real numbers and ¢ is a formal symbol having the property that i? = —1. 


For many years people had “philosophical difficulties” with complex numbers; in- 
deed, numbers of the form ib were called “imaginary,” and the prevailing view was 
that although they did not exist, they were nevertheless very useful. 


From a modern point of view this is neither acceptable (we cannot work with 
things that don’t exist, no matter how useful they may be!) nor necessary: we can 
define the complex numbers entirely in terms of the real numbers. Namely, we may 
identify a complex number a + bi with the ordered pair (a,b) of real numbers, and 
we will define addition and multiplication. Since we would want (a+ bi) +(c+di) = 
(a+c)+(b+d)i, in terms of ordered pairs this is just (a,b) + (c,d) = (a+c,b+d). 
In other words, this is the usual addition of vectors in the plane. The multiplication 
operation is more interesting but still easy enough to write down in terms of only 
real numbers: to compute (a + bi)(c + di), we would want to use the distributive 
law of multiplication over addition and the relation i? = —1. In other words, we 
would like (a + bi) - (ce + di) = ac + bei + adi + bdi? = (ac — bd) + (ad + bc)i. Thus 
in terms of ordered pairs we define a multiplication operation 


(a,b) - (c,d) = (ac — bd, ad + bc). 


Note that with this convention, we may identify real numbers a (i.e., those with 
b = 0) with pairs of the form (a,0); moreover, what we were calling i corresponds 
to (0,1), and now any ordered pair a+ bi can be expressed as (a, 0) + (b, 0) - (0, 1). 


EXERCISE 15.10. Show that the above operations of addition and multiplica- 
tion on ordered pairs satisfy all the field axioms (PO) through (P9). The resulting 
structure is called the complex numbers and denoted C. 


EXERCISE 15.11. Show that because of the relation i? = —1, the field C cannot 
be endowed with the structure of an ordered field. 


Two other important operations on the complex numbers are conjugation and tak- 
ing the modulus. For any complex number z = a+ bi, we define its complex 
conjugate to be Z = a — bi. Conjugation fits in nicely with the rest of the algebraic 
structure: one has 21 + 22 = 21 + Z and (2122) = % 2. 

For any complex number z = a+ bi, we define its modulus (or norm, or abso- 
lute value) to be |z| = Va? +b?. This is just the usual norm of an element of RY 
specialized to the case N = 2. In particular, we have the triangle inequality 


Vz1, 22 € C, lar + 22] < [zi] + |zol. 
However, the norm also behaves nicely with respect to the multiplicative structure. 


PROPOSITION 15.8. a) For all 21, 22 € C, |2122| = |z1||z0|- 
b) For allz €C, 22 =|z|?. 


EXERCISE 15.12. Prove Proposition 15.8. 
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Finally, we mention that the theory of complex series, and especially, of com- 
plex power series, works (at least) as well as the theory of real series. Namely, if 
paar, Gy,2" is a power series with complex coefficients, then defining p = lim sup |a,,| a ‘ 
we find that p is the radius of convergence of the complex power series in the sense 
that the series converges for all z with |z| < p and diverges for all z with |z| > p. 
Especially, if }7,, nz” is a power series with real coefficients and infinite radius of 
convergence, then because for a real number 2, its absolute value || is the same 
as the modulus of the complex number x + 07, then the power seres )7,, dn2” must 
converge for all complex numbers z. 


3. Elementary Functions Over the Complex Numbers 


3.1. The complex exponential function. 


Consider the following complex power series: 


nr 


Zz 
E(z):= ‘S al 
n=0 
1 
Because the ratio test limit is limp_soo oe os = limy-s65 a = 0, the radius of 


convergence is infinite: the series converges for all complex numbers z. 
PROPOSITION 15.9. For all complex numbers z and w, E(z+w) = E(z)E(w). 


PROOF. Since the series representations of E(z) and E(w) are absolutely con- 
vergent, we know that E(z)E(w) is given by the Cauchy product, namely 


Py kwrok ee lee b 5 S(z+tw)” 
=O agent = ate (ale = = et) 
Fomor n=0 k=0 n=0 


Since E(0) = 1, we have for all z that 

E(z)E(-z) = E(z- z) = E(0) =1, 
or E(—z) = Eis) es Note in particular that F(z) is never zero. Restricting attention 
to real values, since E : x +4 E(x) is a continuous function which is never zero and 


such that E(0) = 1, we conclude E(x) > 0 for all real a. 


3.2. The trigonometric functions. 


Let us now turn to the functions sinx and cosa. Recall that we have already shown 
that any pair of differentiable functions S(x) and C(x) such that $’(x) = C(a), 
C(x) = —S(x), S(0) = 0 and C(0) = 1 must be equal to their Taylor series and 
given by the following expansions: 
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Of course we would like to say S(a) = sina and C(x) = cosa, but we do not 
want to have to resort to discussions involving angles, lengths of arcs and other 
such things. We want to see how much can be derived directly from the power se- 
ries expansions themselves. For instance, we would like the show that C? + S$? = 1. 
Unfortunately, although this identity does hold, showing it directly from the power 
series expansions involves some rather unpleasant algebra (try it and see). 


This is where complex numbers come in to save the day: 


PROPOSITION 15.10. For all real x, we have the following identities: 


Gye 5 (Blix) Ee eri 


-(Elix) ~ E(-ia)), 
E(ixz) = C(a) + iS(a). 
EXERCISE 15.13. Prove Proposition 15.10. 


Now we are in business: since the coefficients of E(z) are real, we have E(ix) = 
E(iz) = E(—iz) for all real x, hence 


C(z)? + S(2) = |E(ix)|? = E(ixz)E(ir) = E(ix)E(-ix) = E(ix — iz) = E(0) =1. 
We’re not done yet: we’d like to prove that S(x) and C(x) are periodic functions, 


whose period is a mysterious number approximately equal to 2 - 3.141592653.... 
This can also be worked out from the power series expansions, with some cleverness: 


We first claim that there exists x9 > 0 such that C(ao) = 0. Otherwise, since 
C(0) =1 > 0, we’d have C(x) > 0 for all x, hence S’(x) = C(x) > 0 for all x, hence 
S would be strictly increasing on the entire real line. Since S(0) = 0, it follows that 
S(x) > 0 for all « > 0. Now, if 0 <x < y, we have 

y 


SAGoare i. S(t)dt = C(a) — Cly) <2. 


x 


But now for fixed x and y > 7+ Bay? this gives a contradiction. 


LEMMA 15.11. Let f : [0,co) — R be continuous such that f(0) > 0 and 
f(x) = 0 for some x > 0. Then there is a least positive number xo such that 
f (xo) = 0. 


EXERCISE 15.14. Prove Lemma 15.11. 


Now we define the number 7 by 7 = 2a%9. where po is the lesat positive number x 
such that C(x) = 0. The relation C(x)? + S(x)? = 1 together with C(%) = 0 shows 
that S(5) = +1. On the other hand, since C(x) = $’(x) is non-negative on (0, 5], 
S(z) is increasing on this interval, so it must be that $(Z) = 1. Thus E(#) = i. 
Using the addition formula for E(z) we recover Euler’s amazing identity 


and also e?”’ = 1. In general, e?+?7* = e*e?"' = e”, so E is periodic with period 277i. 


Using the periodicity of F and the formula of Proposition 2, we get that for all 
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x C(x + 2m) = C(a) and S(a+ 2m) = S(z). 


Since for all real t, |e”| = 1, the parameterized curve 
r(t) =e" = C(t) +iS(t) <=> (C(t), S() = (ed), 9) 

has image contained in the unit circle. We claim that every point on the unit circle 
is of the form e” for a unique ¢ € [0,27). To see this, start at the point 1 = e*®, 
and consider ¢ € [0, $]. The function C : [0, $] > R is continuous and decreasing, 
hence injective, with C(0) = 1 and C($) = 0. By the Intermediate Value Theo- 
rem, all values in [0,1] are assumed for a (necessarily unique, by the injectivity) 
t € [0,4], and every point in the first quadrant of the unit circle is of the form 
(x,y) for a unique x € [0,1]. By making similar arguments in the intervals [4,7], 
[w, 22], [32,27] we establish the claim. 


Finally, if we grant that by the arclength of the parameterized curve r(t) = (a(t), y(t)) 
from t = a to t = b we mean the integral 


it is easy to show that C(x) = cosx and S(x) = sinz. Indeed, for r(t) = 
(C(t), S(#)), the arclength integral is 


i S(t) + C2(t)dt = 8, 


=0 
so the point r(@) = (C(@), S(@)) really is the point that we arrive at by starting at 
the point (1,0) on the unit circle and traversing 6 units of arc. 


EXERCISE 15.15. Show that for all x € (0,7/2), S(a) <<a< me 
(Show this directly from the power series definitions: no pictures involving the unit 


circle! Hint: use the power series representation for arctan x.) 


LEMMA 15.12 (DeMoivre). Let k € Z* and z €C. Then there is w € C such 
that w* = z. 


ProoF. If z = 0 we may take w = 0. Otherwise, r := |z| > 0, so 2 lies on 
the unit circle and thus 2 = e”” for a unique 6 € (0,27). We well know (as a 


consequence of the Intermediate Value Theorem) that every positive real number 


Ses L a 1 i8k 
has a positive kth root, denoted r®. Thus if w= r®e’/", 
1 36 6 ; 
w= (re et 7*)k =r(ebF)* = re” =z. 


EXERCISE 15.16. Let z be a nonzero complex number and k a positive integer. 
Show that there are preciesly k complex numbers w such that w* = z. 


4. The Fundamental Theorem of Algebra 
4.1. The Statement and Some Consequences. 
THEOREM 15.13. (Fundamental Theorem of Algebra) Let 
P(z) =anz" +... +012 + 2 


be a polynomial with complex coefficients and positive degree. Then P has a root in 
the comple numbers: there is z € C such that P(z) = 0. 
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Theorem 15.13 is not easy to prove, and we defer the proof until the next section. 
For now we give some important consequences of this seminal result. 


COROLLARY 15.14. Every nonconstant polynomial with complex coefficients 
factors as a product of linear polynomials. More precisely, let 


P(z) =anz" +...+a1z2+ a9, with an #0. 
Then there are a1,...,Q, € C (not necessarily distinct) such that 
(122) P(z) =an(z — a1)(z — a2) +++ (2 — An). 


Proor. First observe that if the result holds for P(z) then it holds for 6P(z) 
for any 6 € C \ {0}. It is therefore no loss of generality to assume that the leading 
coefficient a, of P(z) is equal to 1. Let us do so. 

We now prove the result by induction on n, the degree of the polynomial P. 

Base Case (n = 1): A degree one polynomial with leading coefficient 1 is of the 
form z+ a9 = z — 1, with a, = —ag. 
Induction Step: Let n € Z*, suppose the result holds for all polynomials of degree 
n, and let P(z) be a polynomial of degree n +1. By Theorem 15.13, there is 
zo € C such that P(z)) = 0. By the Root-Factor Theorem, we may write P(z) = 
(z — z0)Q(z), with Q(z) a polynomial of degree n and leading coefficient 1. By 
induction, Q(z) = (z — a1)-+:(z— an), so putting andi = Zo we get 


P(2) = (2-01) +++ (2 an)(Z — Ong): 


More generally, let F' be a field: that is, a set endowed with two binary operations 
denoted + and - and satisfying the field axioms (PO) through (P9) from Chapter 
1. We say that F is algebraically closed if every nonconsant polynomial P(x) 
with coefficients in F has a root in F, i.e., there is x9 € F such that P(ap) = 0. In 
this terminology, the Fundamental Theorem of Algebra asserts precisely that the 
complex field C is algebraically closed. 


EXERCISE 15.17. Let F be an algebraically closed field. Show that the conclusion 
of Corollary 15.14 holds for F': that is, every nonconstant polynomial factors as a 
product of linear polynomials. 


Since every real number is, in particular, a complex number, Corollary 15.14 applies 
in particular to polynomials over R: if P(x) = ana" +...+a1% 4+ ao with a, 4 0, 
then there are complex numbers aj,...,@, such that 


P(a) = a,(a@ — a1)(4% — ag) +++ (@ — ay). 


But since the coefficients of P are real, it is natural to ask whether or to what 
extent some or all of the roots a; must also be real. Recall that we need not have 
any real roots (that is, the field R is not algebraically closed), for any n € Zt, the 
polynomial P,(x) = (2? + 1)” is positive for all real x, so has no real roots. And 
indeed, its factorization over C is (a? +1)" = (a +i)"(a — i)”. 

However, the polynomials P,, all had even degree. Recall that, as a consequence 
of the Intermediate Value Theorem, every polynomial of odd degree has at least 
one real root (and need not have more than one, as the family of examples xP,, (x) 
shows). So there is some relation between the parity (i-e., the evenness or oddness) 
of the degree of a real polynomial and its real and complex roots. This observation 
can be clarified and sharpened in terms of the operation z = a+ bit a— bi = Z of 
complex conjugation. 
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LEMMA 15.15. a) For z,ao,...,@n € C, we have 


poe a 
Anz” +... +12 +a9 =Gnz +...+ 4127+ 40. 


b) Thus, if ao,...,@n € R, we have 


Anz” +... F a,2 +49 = anZ" +... +0a,2 +a. 


c) Let ag,...,@n € R and put P(z) =anz"+...+a12 +49. If 2 € C is such that 
P(z0) =0, then also P(Z) =0. 


PROOF. We have already observed that for any 21, 22 € C, 21 + z2 = 44+ and 
2122 = 22g. Keeping these identities in mind, the proof becomes a straightforward 
exercise which we leave to the reader. 


It is part c) of Lemma 15.15 that is important for us: this well-known result often 
goes by the description “The complex roots of a real polynomial occur in conjugate 
pairs.” To see why this is relevant, consider the following extremely simple — but 
extremely important — result. 


LEMMA 15.16. For a complex number z,z€R = Z=z. 


ProoF. Ifz¢€R, then z = z+0i, soz = z—0i = z. Conversely, let z = a+ bi. 
IFa-bi =Z=2z=a+ bi, then (2i)b = 0. Multiplying through by (2i)~! = =, 
we get b= 0, so z=a+0bdER. 


Let us say that a € C is properly complex if it is not a real number. If P(x) 
is a polynomial with real coefficients and has the properly complex number a as a 
root, then by Lemma 15.16 it also has the (distinct!) properly complex number @ 
as a root. By the Root-Factor Theorem, we may write P(z) = (z—a)(z—@)P2(z). 
Now wake up! — something interesting is about to happen. Namely, if we write 


Pi(2) = (2-a)(z—0), 
then we CLAIM both P;(z) and P2(z) have real coefficients, so we have obtained a 
factorization of real polynomials 
P(x) = P\(x)P2(2). 
For P;(z) we need only write a = a+ bi and multiply it out: 
(z—a)(z —@) = 2% —(a+ @)z + 0G = 2? — (2a)z 4+ (a? + 07). 


For P:(z) we have to argue a bit more abstractly. Namely, for polynomials over R 
(and really, over any field) we can always perform division with remainder: there 
are unique real polynomials Q(x), R(x) such that 


(123) P(x) = Q(x)Pi(x) + R(x), deg R(x) < deg P;(z). 

We claim R(x) = 0 (i-e., it is the zero polynomial). To see this, we use the unique- 
ness part of the division algorithm in a slightly sneaky way: namely, consider P(x) 
and P;(x) as polynomials with complex coefficients and perform the division al- 


gorithm there: there are unique complex polynomials, say Qc(x) and Rc(x), such 
that 


(124) P(x) = Qc(z)Pi(x) + Re(a), deg Rc < deg Qe. 
Here’s the point: on the one hand, real polynomials are complex polynomials, so 


comparing (123) and (124) we deduce Q(x) = Qc(z) and R(x) = Re(x). On 
the other hand, the identity P(x) = P,(x)P2(x) of complex polynomials shows 
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that we may take Qc(x) = Po(x) and Rc(x) = 0. Putting these together, we get 
Q(x) = P(x) and R(x) = 0, so indeed P(x) = P;(x)P2(z) is a factorization of real 
polynomials. 


A positive degree polynomial P() over a field F is called irrreducible if it cannot 
be factored as P(x) = P;(x)P2(x) with deg P;, deg P2 < deg P. (This last condition 
is there to prevent trivial factorizations like x? +1 = (2) - (52? - $-) A polynomial 
of positive degree which is not irreducible is called reducible. 


EXERCISE 15.18. Let F be a field, and let P(x) be a polynomial with coefficients 
in F. 

a) Show that if P is irreducible of degree at least 2, then it has no roots in 
PF. 

b) Suppose P has degree 2 and no roots in F. Show that P is irreducible. 

c) Suppose P has degree 3 and no roots in F. Show that P is irreducible. 

d) For each n > 4, exhibit a degree n polynomial with coefficients in R which 
is reducible, but has no real roots. 


THEOREM 15.17. Let P(x) be a real polynomial of degree n > 1. 


a) There are natural numbers r,s € N such that r+2s = n, linear polynomials 
[y(ax),...,L,(x) and irreducible quadratic polynomials Q1(x),...,Qs(x) 
such that 


P(x) = Ln(2)-+- Lr(@)Qi(x) ++ Qs(2). 
b) If n is odd, then P has at least one real root. 


PrRooF. It’s harmless to assume that P has leading coefficient 1, and we do so. 
a) We go by strong induction on n. When n = 1, P(x) is a linear polynomial, so 
we may take r= n= 1, s = 0 and P(x) = L(x). Suppose n > 2 and the result 
holds for all polynomials of degree less than n, and let P(a) be a real polynomial 
of degree n. By the Fundamental Theorem of Algebra, P(x) has a complex root a. 
If a € R, then by the Root-Factor Theorem P(x) = (« — a)P2(x) and we are done 
by induction. If a is properly complex, Q(x) = (a — a)(x — @) is a real, irreducible 
quadratic polynomial and P(x) = Q(x) P2(x), and again we are done by induction. 
b) Since r + 2s = n, if n is odd we cannot have r = 0. Thus P has at least one 
linear factor and thus at least one real root. 


The partial fractions decomposition rests on the foundation of Theorem 15.17. 


EXERCISE 15.19. (This exercise is taken from http: //math. stackexchange. 
com/ questions/ 1332641/.) Let f(z) =anz"+...+a1z+a9 be a complex polyno- 
mial with positive degree. Suppose that for allz € C, wehavez ER — > f(z) ER. 


a) Show: ao,...,dn ER. 
b) Show: n=1. 


4.2. Proof of the Fundamental Theorem of Algebra. 
We now give a proof of Theorem 15.13, closely following W. Rudin [R, Thm. 8.8]. 


Let P(z) be a polynomial with complex coefficients, degree n > 1, and leading 
coefficient a,. We want to show that P(z) has a complex root; certainly this holds 
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iff + P(z) has a complex root, so it is no loss of generality to assume that the 
leading coefficient is 1 and thus 


P(z) = 1+¢@,_12" + +... +412 + ao, A; (on 


Let 
= inf |P(z)|. 
w= inf |P(z)| 
Thus yz is a non-negative real number and our job is to show (i) that py is actually 


attained as a value of M and (ii) « = 0. 
Step 1: Since for z 4 0, 


it follows that 
lim |P(z)| =0oo-l=oo. 
Zo 


Thus there is R > 0 such that for all z € C with |z| > R, |P(z)| > |P(0)|. By 
Theorem 15.7, the continuous function |P(z)| assumes a minimum value on the 
closed, bounded set {z € C | |z| < R}, say at z. But R was chosen so that 
|P(z)| > |P(0)| > |P(zo)| for all z with |z| > R, so altogether |P(z)| > |P(zo)| for 
all z € C and thus p = infzec |P(z)| = |P(2o)]- 
Step 2: Seeking a contradiction, we suppose 4 > 0. Define Q : C > C by 
Q(z) = epee. Thus Q is also a degree n polynomial function, Q(z) = 1, and by 
minimality of zo, |Q(z)| > 1 for all z € C. We may write Q(z) = 1+b,2*+...+bn2" 
with b, #0 for some 1 <k <n. Let w € C be such that 
zt — —lbxl, 

Sa 

the existence of such a w is guaranteed by Lemma 15.12. Then for r € (0,00), 
Q(rw) =1+ byr® we + bey iret wet +...+b,r"w" 
=1—r* (|by| —rw*ttbpg — 26.7" Fw bp) = 1—r* (\be| + C(r)), 
where we have set C(r) = —rw*t1b,4, —...—r"-*wb,. Thus 
|Q(rw)| = [1 —r¥([be| + C(r))| < [b= 1" bel] + Ir*C(r)]. 
As r approaches 0 from the right, r*|b,| and C(r) both approach 0. Thus for 
sufficiently small r, we have r*|b;,| <1 and |C(r)| < |by| and then 
|Q(rw)| < [Lr [bel + [r®C(r)| = 1 r*([bu| = |C(r)I) < 1 

This contradicts the fact that minzec |Q(z)| = 1 and completes the proof. 


CHAPTER 16 


Foundations Revisited 


The reader should picture a street mime juggling non-existent balls. As the mime 
continues, the action of juggling slowly brings the balls into existence, at first in 
dim outline and then into solid reality. —T.W. Korner? 


An ordered field F is Dedekind complete if every nonempty subset which is 
bounded above has a least upper bound (or “supremum” ). 


EXERCISE 16.1. Show that an ordered field is Dedekind complete iff every 
nonempty subset which is bounded below has a greatest upper bound (or “infimum”). 


Our initial definition of R was precisely that it was a Dedekind complete ordered 
field. Practically speaking, this is a great foundation for honors calculus and real 
analysis, because it contains all the information we need to know about R. 


In other words, we have put a neat little black box around our foundational prob- 
lems. Real analysis works perfectly well without ever having to look in the box. 
But curiosity is a fundamental part of mathematics, and at some point most of us 
will want to look in the box. This chapter is for those who have reached that point, 
i.e., who want to understand a proof of the following theorem. 


THEOREM 16.1. (Black Box Theorem) 
a) There is a Dedekind complete ordered field. 
b) If F, and F2 are Dedekind complete ordered fields, they are isomorphic: that is, 
there is a bijection ® : F, — F2 such that: 
(i) For allz,y € Fi, O(a + y) = O(a) + B(y). 
(ii) For all x,y € Fi, ®(ay) = ®(x) ®(y). 
(iti) For allz,ye Fi, r<y <> (2) < @(y). 
c) The isomorphism ® of part b) is unique: there is exactly one such map between 
any two Dedekind complete ordered fields. 


The Black Box Theorem explains why we never needed any further axioms of R 
beyond the fact that it is a Dedekind complete ordered field: there is exactly one 
such structure, up to isomorphism.” 


1Thomas William Korner, 1946— 

2The student unfamiliar with the notion of “isomorphism” should think of it as nothing else 
than a relabelling of the points of R. For instance consider the z-axis Rz = {(a,0) | a € R} 
and the y-axis Ry = {(0,a) | a € R} in the plane. These are two copies of R. Are they “the 
same”? Not in a hard-nosed set-theoretic sense: they are different subsets of the plane. But 
they are essentially the same: the bijection ® which carries (a,0) ++ (0,a) preserves the addition, 
multiplication and order relation. So really we have two slightly different presentations of the same 
essential structure. An arbitrary isomorphism is no more than this, except that the “relabelling 
map” ® might be more complicated. 
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We will prove the Black Box Theorem...eventually. But rather than taking the 
most direct possible route we broaden our focus to a study of the structure of 
ordered fields, not just Q and R. 


1. Ordered Fields 


1.1. Basic Definitions. 


In this section we revisit the considerations of §1.2 from a somewhat different per- 
spective. Before we listed certain ordered field axioms, but the perspective there 
was that we were collecting true, and basic, facts about the real numbers for use in 
our work with them. This time our perspective is to study and understand the col- 
lection of all ordered fields. One of our main goals is to construct the real numbers 
R in terms of the rational numbers Q and to understand this in terms of a more 
general process, completion, which can be applied in any ordered field. 


A field is a set F’ endowed with two binary operations + and - which satisfy 
all of the field axioms (PO) through (P9). To a first approximation, these axioms 
simply encode the usual rules of addition, subtraction, multiplication and division 
of numbers, so any field can be thought of as a kind of “generalized number sys- 
tem”. The most important basic examples are the rational numbers Q, the real 
numbers R, and the complex numbers C. But there are other examples which seem 
farther removed from the “usual numbers”: e.g. finite fields like F, = {0,1} are 
smaller than what we normally think of as a number system, whereas the set R(t) of 
all rational functions (with real coefficients) is a field whose elements are naturally 
regarded as functions, not as numbers. 


Field theory is an active branch of mathematical research, with several texts and 
thousands of papers devoted to it (e.g. [FT]). Nevertheless the very simple prop- 
erties of fields established in § 1.2.1 will be sufficient for our needs here, in part 
because we are not interested in fields per se but rather ordered fields. An ordered 
field is a field equipped with the additional structure of a total order relation, 
namely a binary relation < which satisfies: 


(TO1) Reflexivity: forallaeF,a<a. 

(TO2) Antisymmetry: for allz,y € F,ifa<yandy <a, thenz=y. 
(TO3) Transitivity: for all 2, y,z € F,ifa<y and y < z then 2 < z. 
(TO4) Totality: for all x,y € F, either <yory <a. 


Given a total order, we may define a strict ordering x < y by « < y but x Z y. 
This is (as we well know!) natural and useful. From an abstract perspective the 
key remark is that it is essentially equivalent to a total ordering. Namely, any strict 
ordering coming from a total ordering satisfies the following modified versions of 
the above axioms: 


(TR) Trichotomy: for all x,y € F, exactly one of the following holds: 


L<Y T=y; or L> y. 
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(TO3): Transitivity: for all z,y,z € F, if <yand y < z then x < z. 


EXERCISE 16.2. Let (F,<) be a total order relation on the set F’. 


a) Show that the associated strict ordering satisfies (TR) and (TOS). 

b) Let < be a binary relation on F satisfying (TR) and (TO3). Definex < y 
byx<y orxz=y. Show that this gives a total ordering on F. 

c) Show that the processes of passing from a total ordering to its strict order- 
ing and from a strict ordering to its total ordering are mutually inverse: 
doing one followed by the other, in either order, brings us back where we 
started. 


In light of the preceding exercise we may (and shall) in fact pass from a total or- 
dering to its strict ordering and back without explicit comment. 


An ordered field is a field (F,+,-) equipped with a total ordering < which is 
compatible with the field structure in the sense of satisfying the following two 
familiar axioms: 


(P11) If x,y > 0, then e+ y > 0. 
(P12) If x,y > 0, then ry > 0. 


We refer the reader back to § to 1.2.2 for the most elementary consequences of 
these axioms, e.g. Proposition 1.11. 


Now we introduce a new idea. Let F and F” be ordered fields. (To be formally 
corrrect, we should speak of “the ordered field (F,+,-,<)”, and similarly for F’. 
In practice this extra notation weighs us down without any advantages in clarity 
or precision.) An ordered field homomorphism is a map f : F —> F” satisfying 
all of the following: 


(FH1) For all z,y € F, f(x +y) = f(x) + fly). 
(FH2) For all x,y € F. f(xy) = Fl) f(). 
(FHB) f() = 1 

(OFH) For all x € F, if x > 0, then f(x) > 


EXERCISE 16.3. Show that for any homomorphism of ordered fields, we have 
f(0) =0. 


Because of Exercise 16.3 it is natural to wonder whether we really need (FH3) or 
whether in fact f(1) = 1 follows automatically from the other axioms. The answer 
is a resounding no. Indeed, in the absence of (FH3), for any ordered fields F' and 
F" we could get a homomorphism betweeen them simply by mapping every x € F’ 
to 0. This is not an interesting map,® and as we shall see the theory is cleaner for 
not allowing it. 


If F and F” are fields, then a field homomorphism is a map f : F > F” sayisfying 
(FH1) through (FH3). 


PROPOSITION 16.2. Every field homomorphism f : F > F’ is injective. 


3Nothing can come of nothing. — King Lear 
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Proor. Assume to the contrary that there are x 4 y € F with f(x) = f(y). 
Then f(a — y) = f(x) — f(y) = 0. Since « 4 y, x — y # O, and thus we have a 
multiplicative inverse a Then 


1 1 1 


1 =f) = F(@-W—) = fle- KG) = 0-H) = 0. 


contradicting axiom (PO). Note the use of (FH3)! 


A field isomorphism is a field homomorphism f : F' > F’ which has an inverse: 
i.e., there is a field homomorphism f’ : F’ > F such that fof =1p, fof’ =1p. 
EXERCISE 16.4. Show that for a field homomorphism f : F — F", the following 
are equivalent: 
(i) The map f is a field isomorphism. 
(ii) The map f is bijective.* 
(iii) The map f is surjective. 
EXERCISE 16.5. Let f : F > F’ be a field homomorphism. Show that f induces 
a field isomorphism F + f(F). 


A subfield of a field & is a subset of K which is a field under the binary operations 
+ and-on K. For example, Q is a subfield of R and R is a subfield of C. 


EXERCISE 16.6. Let K be a field and F a subset of K. Show that F is a subfield 
iff all of the following hold: 
(SF1) We have 0,1 € F. (In particular F is nonempty.) 
(SF2) Foralla,yeF,x+y,c—y, ry € F. 
(SF3) If € F andx 40, thent+e F. 


Thus whenever we have a field homomorphism f : F’ — F’, we get an isomorphic 
copy of F' as a subfield of F’, namely f(F). Because of this it is safe to think of of 
F itself as a subfield of F’, and this perspective is often useful. 


EXERCISE 16.7. Let K be an ordered field, and let F C K be a subfield. Show 
that restricting the total ordering < on K to F endows F with the structure of an 
ordered field in such a way that the inclusion map .: F + K is a homomorphism 
of ordered fields. 


We say that an element a of a field F is a sum of squares if there is n € Z*+ and 
£1,-..,2n € F such that a = xi +...+ 22. Notice that in any field 0 and 1 are 
sums of squares. 


EXERCISE 16.8. Let F' be a field. 


a) Let x € F be a sum of squares. Show that for any compatible ordering < 
on F,x>0. 

b) Suppose that there is a nonzero element x € F such that both x« and —x 
are sums of squares. Show that there does not exist an ordering < on F 
compatible with the field structure. 


4Some philistines take this as the definition of a field isomorphism. Without getting into 
too much detail about it, this is really the wrong definition. It happens to be equivalent to the 
right definition for fields, but it has an analogue for other types of isomorphisms of mathematical 
structure which is not always true. E.g. this definition suggests that an isomorphism of topological 
spaces should be a continuous bijection, and this is truly weaker than the correct definition, namely 
a continuous map between topological spaces which admits a continuous inverse. 
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c) Deduce that the field C of complex numbers admits no compatible ordering. 


EXERCISE 16.9. Let F' be a field. 


a) Suppose that F has the following property: for each nonzero x € F’, exactly 
one of x and —x is a sum of squares. Show that F admits a unique 
compatible ordering: namely x > 0 iff x is a sum of squares. 

b) Show that the standard ordering on R is the only compatible ordering. 

c) Show that the standard ordering on Q is the only compatible ordering. 


EXERCISE 16.10. A field F is formally real if —1 is not a sum of squares in 


a) Suppose that F admits a compatible ordering. Show that F is formally 
real. (Hint: show the contrapositive.) 

b) We suppose that1+140€ F. (If not, then —1 =1 is a sum of squares, 
so by part a) F admits no compatible ordering.) remarkable theorem of 
Artin-Schreier asserts for x € F, the following are equivalent: 

(i) For every compatible ordering < on F, x <0. 

(ii) The element x is a sum of squares. Deduce from this theorem that 
if F is formally real then it admits at least one ordering. (Hint: (i) 
holds vacuously if there are no compatible orderings!) 


PROPOSITION 16.3. For every ordered field F’, there is a unique ordered field 
homomorphism v:Q—-> F. 


PROOF. Step 1: Since F can be ordered, the sum of positive elements is pos- 
itive: in particular the sum of positive elements is not zero. Thus for any positive 
integer n, 1+...+ 1 (n times) is not zero. The map ¢ which sends 0 to 0 and 
each positive integer to u(n) = 1+...+ 1 (n times) therefore gives an injective 
map N + F. We can extend this map to negative integers by mapping —n to the 
additive inverse of 1+...+ 1 (n times). Further, for each positive integer n, since 
L(n) # 0, it has a multiplicative inverse noe and we may map @ ++ um) ay: It is 
now straightforward (but not completely trivial) to check that this map 1: Q—- F 
is a field homomorphism: we leave this verification to the reader. 

Step 2: In fact we had no choice in the matter: the map uz is the unique field ho- 
momorphism from Q to F’. We leave this to the reader. 

Step 3: We must check that if * = ™ € Q is positive, then so is u(a) = u(m) zy: 
But this is easy: u(m) is 1+...+ 1, so it is a sum of positive elements and is 
thus positive. Similarly 1(n) is positive, and thus so is its reciprocal aay Finally, 


um) = u(m) zy is a product of two positive elements, hence positive. 


In light of this result, for any ordered field F', we may view Q as a subfield. 


1.2. Some Topology of Ordered Fields. 


Let (4, <) be a linearly ordered set, and let & C K be a subset. We say that 
F is cofinal in K if for all x € K, there is y € F with y > x. In fact this is pre- 
cisely the concept that in the case of subsets of R we call unbounded above. We 
use this terminology as an indication that this property can behave a bit differently 
in a non-Archimedean field. 
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By definition, an ordered field K is Archimedean if Z is cofinal in K. Equiva- 
lently, K is Archimedean iff Q is cofinal in K. 


Let F be a subfield of the ordered field fF’. We say that F is dense in K if 
for alla < y € K, there is z € F such that r<z<y. 


LEMMA 16.4. Let K be an ordered field, and let F be a subfield of K. If F is 
dense in K, then F is cofinal in K. 


PROOF. We show the contrapositive: suppose F’ is not cofinal in Kk: there is 
x € K such that for all y € F, y < x. Then the interval (2,2 + 1) contains no 
points of F’, so F is not dense in K. 


More generally, let f : fF > F’ be a homomorphism of ordered fields. We say that 
f is cofinal if the image f(F’) is a cofinal subfield of F’. We say that f is dense 
if the image f(F’) is a dense subfield of F’. 


EXERCISE 16.11. Let K be a subfield of F. 


a) Suppose that for every a € F, there is a sequence {x,} of elements of K 
such that x, + a. Show that K is a dense subfield of F. 

b) Does the converse of part a) hold? 
(Hint: no, but countereramples are not so easy to come by.) 


LEMMA 16.5. For a homomorphism f : F — F' of ordered fields, the following 
are equivalent: 
(i) The map f is cofinal. 
(ii) For every positive €' € F’, there is a positive 6 € F such that f(d) < é. 
EXERCISE 16.12. Prove Lemma 16.5. (Hint: take reciprocals!) 


LEMMA 16.6. For an ordered field F’, the following are equivalent: 


(i) F is Archimedean. 
(ii) Q is a dense subfield of F. 


PRooF. (i) == (ii): Suppose F’ is Archimedean and let « < y € F. Let 
n € Zt be such that 7 <n; then 0< 4 <y-— 2, s0 


1 
E<er+—<y. 
n 


(ii) — > (i): If Q is dense in F then by Lemma 16.4, Q is cofinal in F. 


Let S be a subset of an ordered field F. For s € S, we say that S is discrete 
at s if there is a positive e € F such that (s—e,s+e¢) 1S = {s}. We say that 
S is discrete if it is discrete at s for all s € S. An ordered field homomorphism 
f:F — F’ is discrete if f(F) is a discrete subset of F’. 

PROPOSITION 16.7 (Dorais’s Dichotomy). Let f : F — F” be an ordered field 
homomorphism. Then exactly one of the following holds: 


(i) The map f is cofinal. 
(ii) The map f is discrete. 


PROOF. We will use the characterization of cofinality from Lemma 16.5. 
Suppose f is cofinal. Then 0 € f(F') and for every positive «’ € F’, there is a 
positive 6 € F such that f(5) € (—e,e). It follows that f(£’) is not discrete. 
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Suppose f is not cofinal: there is e’ € F” such that (—e’,e)N f(F) = {0}: f(F) 
is discrete at 0. Now let x € F’. Suppose there is y € F' such that r—€ < y< a+e. 
Then y — x € (€,€). By what we’ve just seen we must have y— x2 = 0, ie, y= 2. 
Thus f(£) is discrete at each of its elements, so it is discrete. 


For any element x in an ordered field, we can define || in the usual way, i.e., x if 
x >0Oand —-2 ifx <0. 


EXERCISE 16.13. Let f : F — F”’ be an ordered field homomorphism. Show 
that for all x € F, |f(x)| = f(|z|). (The absolute value on the left hand side is 
taking place in F’; the one on the right hand side is taking place in F.) 


It is now possible to carry over our definitions of convergent sequencess, Cauchy 
sequences and continuous functions to the context of ordered fields and homomor- 
phisms between them. We repeat the basic definitions. In many cases the proofs 
are exactly the same as in the case F’ = R treated in loving detail in this text; when 
this is so we leave the proofs to the reader. However, there are some things which do 
not carry over to the context of all ordered fields, and we treat these in some detail. 


First recall that for any set S, a sequence {z,} in S is given by a function f : 
Z* — S; we write x, in place of f(n). 


Let {x,,} be a sequence in an ordered field F and let x € F'. We say that 2, 
converges to x and write x, — 7 if: for all positive « € F, there is N € Z* such 
that for alln > N, |tp, — 2| < €. We say that a sequence {z,,} is convergent if 
In — x for some x € F. 


We observe that this is verbatim the same as the definition in R. At the same 
itme, the fact that our “small €” is now an element of the ordered field F’ rather 
than (necessarily) a real number has certain surprising implications. The following 
exercises exhibits one. 


EXERCISE 16.14. For an ordered field F’, show that the following are equivalent: 


(i) The sequence {4+}°°., is convergent in F. 
(ii) The ordered field F is Archimedean. 


LEMMA 16.8. Let {x} be a sequence in an ordered field F, and let x,y € F. 
ft, x and zy, > y, thenx = y. 


EXERCISE 16.15. Prove Lemma 16.8. 


Thus if z, — x we may call x the limit of the sequence: it is unique. 


A sequence {x,,} in an ordered field F' is Cauchy if for all positive « € F there is 
N €Z* such that for all m,n > N, |@m —n| < €. 


PROPOSITION 16.9. A Cauchy sequence in an ordered field which admits a 
convergent subsequence is itself convergent. 


Proor. Let F be an ordered field, let {x,,} be a Cauchy sequence in F’, and 
let {n,,} be a subsequence converging to x € F. Let € > 0. Since {x,} is Cauchy, 
there is N € Z* such that for all m,n > N, |@m — 2n| < a Also there is kK > N 
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such that for all k > K, |an, —a|< §. Ifn > K, then since nx > K we have 


€ € 
[Zn — Z| < [fn — Eny| + |Pny — Z| < r>=6€, 


2 2 


SO Ln > 2. 


LEMMA 16.10. A convergent sequence in an ordered field is Cauchy. 
EXERCISE 16.16. Prove Lemma 16.10. 


On the other hand, in an arbitrary ordered field a Cauchy sequence need not be 
convergent. For instance the “Babylonian sequence” {x,,} of §VII.2.2 is a sequence 
of rational numbers converging to 2 € R. Since V2 ¢ Qand limits in R are unique, 
viewed as a sequence in the ordered field Q, the sequence {x,,} is not convergent. 
It is however Cauchy. More generally we have the following result. 


PROPOSITION 16.11. Let f : F — F”’ be an ordered field homomorphism, and 
let {ty} be a sequence in F’. If the sequence {f(tn)} is Cauchy in F’, then {x,} 
is Cauchy in F. 

PrRooF. We show the contrapositive: suppose {2,,} is not Cauchy in F’. Then 
there is some positive « € F such that for all N € Z* there are m,n > N such that 
lfm —%n| > €. Then 

f() S F(lam — nl) = |f(@m — n)| = |F (tm) — F(@n)I- 
Since ¢’ = f(e) is a positive element of F’, this shows that {f(x,)} is not Cauchy. 


Thus we get a method for producing nonconvergent Cauchy sequences in an ordered 
field F: find a sequence {x,} in F and a homomorphism f : F > F” such that 
f (an) converges to an element of F’ \ f(F). In fact every nonconvergent Cauchy 
sequence arises this way, as we will see later on. 


Let F and F” be ordered fields, and let f : F — F’ be a map between them. 
We say that f is continuous at c € F if for all positive «’ € F’, there is a pos- 
itive 6 € F such that for all x € F with |x — c| < 6, |f(x) — f(c)| < &. We say 
that f : F — F’ is continuous if it is continuous at every c € F’. We say that 
f:F > F’ is uniformly continuous if for all positive «’ € F” there is a positive 
6 € F such that for all z,2’ € F, if |x — 2'| < 6 then |f(x) — f(2’)| < €. 


LEMMA 16.12. Let f : F > F’ be a map between ordered fields. 


a) If f is continuous and x, is a sequence in F' which converges to x € F, 
then f(an) > f(a) in F’. 

b) If f ts uniformly continuous and {x,} is a Cauchy sequence in F’, then 
{f(an)} is a Cauchy sequence in F’. 


EXERCISE 16.17. Prove Lemma 16.12. 


Warning: When F = F’ = R, the converse of Lemma 16.6a) holds (Theorem 10.5): 
a map which preserves limits of convergent sequences is necessarily continuous. This 
does not hold in general; but unfortunately counterexamples lie beyond the scope 
of this text. 


THEOREM 16.13 (F. Dorais). Let f : F — F’ be a homomorphism of ordered 
fields. The following are equivalent: 
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(i) The map f is uniformly continuous. 
(ii) The map f is continuous. 
(iii) The map f is continuous at 0. 

(iv) The map f is cofinal. 

PROOF. (i) => (ii) => (iii) is immediate. 
(iii) <= (iv): By Lemma 16.5, f(£) is cofinal in F’ iff for every positive é’ in 
F’, there is a positive 6 in F such that f(6) < ¢’. Suppose this holds, and let 
e’ be positive in F’, and choose a positive 6 € F' as above. Then for x € F, if 
|x| = |x — 0| < 6, then 

f(z) = F(lel) < £6) = |F() — FO) < €. 

So f is continuous at 0. Conversely, if f is continuous at 0, then for each positive 
é € F there is a positive 6 € F such that if |x| < 6, |f(a)| < . Thus 2 is a positive 
element of F’ such that £9) <<’, so f(F”) is cofinal in F by Lemma 16.5. 
(iii) => (i): Let é’ be a positive element of F’; since f is continuous at 0 there is 
a positive 6 € F such that for all « € F with |z| < 6, |f(x)| < é. So, ifa,yeF 
are such that |x — 2’| < 6, then 


If(x) — fl =Ifl@—y)| < €. 
1.3. A Non-Archimedean Ordered Field. 


Let K = R((t)) be the field of formal Laurent series with R-coefficients: an 
element of K is a formal sum )°,,¢z ant” where there exists N € Z such that 
dy, = 0 for all n < N. We add such series term by term and multiply them in the 
same way that we multiply polynomials. 


EXERCISE 16.18. Show that K = R((t)) is a field. 


We need to equip K with an ordering; equivalently, we need to specify a set of pos- 
itive elements. For every nonzero element « € K, we take u(x) to be the smallest 
n € Zsuch that a, 4 0. Then we say that « is positive if the coefficient a, .) of 
the smallest nonzero term is a positive real number. It is straightforward to see 
that the sum and product of positive elements is positive and that for each nonzero 
x € K, exactly one of x and —z is positive, so this gives an ordering on K in the 
usual way: we decree that x < y iff y — x is positive. 

We observe that this ordering is non-Archimedean. Indeed, the element + is 
positive — its one nonzero coefficient is 1, which is a positive real number — and 
infinitely large: for any n € Z, + — nis still positive — recall that we look to the 
smallest degree coefficient to check positivity — so + >n for all n. 

Next we observe that the set {+} is unbounded in Kk. Taking reciprocals, it 
follows that the sequence {t”} converges to 0 in K: explicitly, given any « > 0— here 
€ is not necessarily a real number but any positive element of F'! — for all sufficiently 
large n we have that 7 > 4, so |t”| =" <. We will use this fact to give a simple 
explicit description of all convergent sequences in F’. First, realize that a sequence 
in K consists of, for each m € Z* a formal Laurent series 2, = Yonez Qm,nt”, So in 
fact for each n € Z we have a real sequence {@m,n}°0_,. Now consider the following 
conditions on a sequence {z,,} in K: 


(i) There is an integer N such that for all m € Zt and n < N, adm» = 0, and 
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(ii) For each n € Z the sequence amp is eventually constant: i.e., for all sufficiently 
large M, @mn = Cn € R. (Because of (i) we must have C,, = 0 for all n < N.) 


Then condition (i) is equivalent to boundedness of the sequence. 


I claim that if the sequence converges — say 2m — © = Dy Gnt” € F — then (i) 
and (ii) both hold. Indeed convergent sequences are bounded, so (i) holds. Then for 
alln > N, am.n is eventually constant in m iff dm —Gn is eventually constant in m, 
so we may consider x,, —x instead of x, and thus we may assume that xz, > 0 and 
try to show that for each fixed n, dm» is eventually equal to 0. As above, this holds 
iff for all k > 0, there exists M;, such that for all m > Mg, |am| < t*. This latter 
condition holds iff the coefficient ay, of t” in x, is zero for all N < k. Thus, for all 
m > Mk, Qm,—-N = Om,-N+1 = +++ = Gm,k—1 = 0, which is what we wanted to show. 


Conversely, suppose (i) and (ii) hold. Then since for all n > N the sequence 
Qmn is eventually constant, we may define a, to be this eventual value, and an 
argument very similar to the above shows that r, 4 @ = 7,3. Gnt”. 


Next I claim that if a sequence {x,,} is Cauchy, then it satisfies (i) and (ii) above, 
hence is convergent. Again (i) is immediate because every Cauchy sequence is 
bounded. The Cauchy condition here says: for all k > 0, there exists M;, such 
that for all m,m’ > My, we have |x, — x/,| < t®, or equivalently, for all n < k, 
GQm,n — mn = 0. In other words this shows that for each fixed n < k and all 
m > My, the sequence a@m,n is constant, so in particular for all n > N the sequence 
Gm,n is eventually constant in m, so the sequence x, converges. 


EXERCISE 16.19. Show that the subfield R((t?)) of R((t)) is cofinal but not 
dense. 


2. The Sequential Completion 
2.1. Sequentially Complete Fields. 


An ordered field F' is sequentially complete if every Cauchy sequence in F' con- 
verges to an element of F'. We have seen that a Dedekind complete ordered field is 
sequentially complete. Here we wish to examine the converse. 


THEOREM 16.14. For an Archimedean ordered field F, TFAE: 
(i) F is Dedekind complete. 
(ii) F is sequentially complete: every Cauchy sequence converges. 


Proor. The implication (i) => (ii) is the content of Theorem 10.32, since 
the Bolzano-Weierstrass Theorem holds in any ordered field satisfying (LUB). 
(ii) = > (i): Let S Cc F be nonempty and bounded above, and write U(S) for 
the set of least upper bounds of S. Our strategy will be to construct a decreasing 
Cauchy sequence in U/(S) and show that its limit is sup S. 

Let a € Sand b € U(S). Using the Archimedean property, we choose a negative 
integer m < a and a positive integer M > b, so 


m<a<b<M. 
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For each n € Zt, we define 
k 
S,={keZ| an: €U(A) and k < 2"M}. 


Every element of S,, lies in the interval [2"m,2"M] and 2”M € S,, so each S,, is 


finite and nonempty. Put k, = min S, and a, = Ka sO see = he U(S) wihle 


2k? = knot ¢ U(S). It follows that we have either kn41 = 2kn or kn41 = 2kyn-1 
and thus either @,41 = Gy OY An41 = Gn — sur. In particular {a,,} is decreasing. 


For all 1 <m <n we have 


0 < am — an = (dm Am-+1) (Qm-41 Am +2) Feeed (n—1 An) 
SE or a 

Thus {a,,} is Cauchy, hence by our assumption on F' a, > L € F. 

We CLAIM L = sup(S). Seeking a contradiction we suppose that L ¢ U(S). 
Then there exists x € S such that L < x, and thus there exists n € Z* such that 

Gn — L =|a, —L| <a—L. 

It follows that a, < x, contradicting a, € U(S). So L € U(S). Finally, if there 
exists L’ € U(S) with L’ < L, then (using the Archimedean property) choose 


né Zt with x < L—L’, and then 


1 1 
ee Sh Se Se 
Oe on = 9 
SO An — x = fad € U(S), contradicting the minimality of ky. 


The proof of (ii) = > (i) in Theorem 16.14 above is taken from [HS] by way of 
[Ha11]. It is rather unexpectedly complicated, but I do not know a simpler proof 
at this level. However, if one is willing to introduce the notion of convergent and 
Cauchy nets, then one can show first that in an Archimedean ordered field, the 
convergence of all Cauchy sequences implies the convergence of all Cauchy nets, 
and second use the hypothesis that all Cauchy nets converge to give a proof which 
is (in my opinion of course) more conceptually transparent. This is the approach 
taken in my (more advanced) notes on Field Theory [FT]. 


In fact there are (many!) non-Archimedean sequentially complete ordered fields. 
We will attempt to describe two very different examples of such fields here. We 
hasten to add that this is material that the majority of working research mathe- 
maticians are happily unfamiliar with, and which is thus extremely rarely covered 
in undergraduate courses. Only the exceptionally curious need the next section. 


2.2. Sequential Completion I: Statement and Applications. 


We will now establish one of our main results: for every ordered field F’, there 
is a sequentially complete ordered field R and a homomorphism f : F > R. 


In fact we can, and will prove, even more than this. The point is that there will 
be many (nonisomorphic) sequentially complete fields into which any given ordered 
field embeds. For example, when we construct the real numbers we will have an 
embedding Q <> R. But we also have an embedding R ~ R((¢)), so taking the 
composite gives an embedding Q — R((t)). (There is no way that R and R((¢)) are 
isomorphic, since the former is Archimedean and the latter is not.) 
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We would like a general definition which allows us to prefer the embedding 
Q > R to the embedding Q ~ R((t)). The key observation is that, since R is 
Archimedean, the embedding of Q into R is dense, whereas since R((t)) is not 
Archimedean, the embedding of Q into R is not dense. This leads to the following 
important definition. 


A sequential completion of an ordered field F’ is a dense embedding F ~~ R 
into a sequentially complete ordered field. 


LEMMA 16.15. For an ordered field F’, the following are equivalent. 
(i) F is Dedekind complete. 
(it) The inclusion t: Q— F makes F into a sequential completion of Q. 


Proor. (i) = > (ii): By Theorem 16.14, F is sequentially complete and 
Archimedean. By Lemma 16.6, Q = +(Q) is a dense subfield of F’, and it follows 
that F is a sequential completion of Q. 


We will prove that every ordered field admits a sequential completion. And again, 
we will in fact prove a bit more. 


THEOREM 16.16. Let F' be an ordered field. 
a) F admits a sequential completion: F > R. 
b) If L is any sequentially complete ordered field and f : F — L is a cofinal ordered 
field homomorphism, then there is a unique ordered field homomorphism g: R > L 
such that f = gov. 


COROLLARY 16.17. Two sequential completions of the same ordered field are 
isomorphic. 


PrRoor. Let 1; : F > Ry, and tg : F + Re be two sequential completions. 
Applying Theorem 16.16 with R = R, and f = 12: F > Ro, we get a unique 
homomorphism g : Ry — Rez such that v2 = gov,. Interchanging the roles of Ry 
and R»z we also get a unique homomorphism g’ : R2 + Ry, such that 1) = g/ oL2. 

Now consider g'0g: Ri 4 Roo. We have 

(g'og)ou =g'o(gou) =g' ot = 41. 
Applying Theorem 16.16 with L = R = R, we get that there is a unique homomor- 
phism G: Ro + Ri such that Gov = 1, but clearly the identity map 1p, has this 
property. Thus we must have g’ og = 1R,. Similarly considering go g’ : Re > Ra, 
then in view of 

(gog') or =9o(g' ot) =gou = la, 
we deduce that gog’ = 1R,. In other words, g and g’ are mutually inverse isomor- 
phisms...so R, and Rz are isomorphic. 


Applying Theorem 16.16 to the ordered field Q, we get a sequential completion R 
of Q. Since R is Archimedean and sequentially complete, by Theorem 16.14, R is 
Dedekind complete. Conversely, by Lemma 16.15any Dedekind complete ordered 
field R’ is isomorphic to R. Thus the existence and uniqueness statements of The- 
orem 16.16 imply the existence and uniqueness up to isomorphism of a Dedekind 
complete ordered field. 


The uniqueness statement can be strengthened: let R, and 2 be two Dedekind 
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complete ordered fields. We claim that not only are they isomorphic, but that the 
isomorphism between them is unique. Indeed, for 7 = 1,2 let 1; : Q > R; be the 
inclusion maps. We saw above that there is a unique map g : Ri — Re such that 
gl, = lg and this g is an isomorphism. But any isomorphism h : Ri > Rea 
will satisfy h o 1, = lg, since in fact there is exactly one embedding from Q into 
any ordered field. Thus whereas in general there is an isomorphim g between two 
sequential completions of a given ordered field F’ which is unique such that blah 
blah blah (more precisely, such that go 11 = 2), in this case the “blah blah blah” 
is vacuous and the isomorphism is unique full stop. 


In abstract mathematics, uniqueness up to a unique isomorphism is as close to 
identical as we can reasonably ask for two structures to be. (Even the “horizontal 
copy of R” and the “vertical copy of R” are different sets, but the obvious isomor- 
phism between them is the only isomorphism, so no trouble can arise by identifying 
the two.) We denote this unique field by R and call it the real numbers...of course. 


2.3. Sequential Completion II: The Proof. 


Now we are properly motivated to roll up our sleeves and endure the rather lengthy, 
technical proof of Theorem 16.16. The essential idea (which is indeed due to A.L. 
Cauchy) is to build the sequential completion directly from the set C of all Cauchy 
sequences in F’. 


We can observe that C itself has some structure reminiscent of an ordered field 
but that things do not quite work out: it is somehow too large to itself be an or- 
dered field. Namely, it makes perfectly good sense to add, subtract and multiply 
Cauchy sequences in F’. For that matter, it makes perfectly good sense to add, 
subtract and multiply arbitrary sequences in F’: we simply put 


{tn} + {yn} = {En + Yn}, 
{tn} — {Yn} = {fn — Yn}, 
{in} + {Yn} = {ln + Yn}. 


It remains to check that these operations take Cauchy sequences to Cauchy se- 
quences. At the very beginning of our study of sequences we showed this for 
convergent sequences (in R, but the proofs certainly did not use any form of the 
completeness axiom). It is no more difficult to establish the analogue for Cauchy 
sequences in F’. 


LEMMA 16.18. Let F' be any ordered field, and let a., b. be Cauchy sequences. 
Then ag +b. and a,- be are both Cauchy. 


PRoor. Since a, and b, are both Cauchy, for e > 0 there is N € Z such that 
for m,n > N, |am — an| < € and and |bm — bn| < ¢. Then 


l(Q@m + bm) — (@n + On)| < |Qm — Qn] + |Wm — bn| < 2e. 


Further, since the sequences are Cauchy, they are bounded: there are M,, My € F 
such that |an| << Ma and |b,| < M;, for all n € Z*. Then for m,n >N, 


|GmDm a AnD | < |Qmn > Gn||Pn| =e |an||Om = br| < (Ma ate My)e. 
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So does this addition and multiplication endow C with the structure of a field? 
There is an additive identity, namely the sequence with x, = 0 for all n. There 
is also a multiplicative identity, namely the sequence with x, = 1 for all n. It all 
works well until we get to multiplicative inverses. 


EXERCISE 16.20. Let {x,} be a sequence in the ordered field F. 
a) Show that there is a sequence {yn} with {an} - {yn} = {1} if and only if 
for alln€ Z*, x, #0. 
b) Show that if {a} is Cauchy and x, #0 for all n, then its inverse {+} 
is again a Cauchy sequence. . 


Thus there are plenty of Cauchy sequences other than the constantly zero sequence 
which do not have multiplicative inverses: e.g. (0,1,1,1,...), or indeed any constant 
sequence which takes the value 0 at least once. Thus C has many good algebraic 
properties, but it is not the case that every nonzero element has a multiplicative 
inverse, so it is not a field.® 


We also have some order structure on C. For instance, it is tempting to define 
{tn} > {Yn} if Zp > Yn for all n. This turns out not to be a good definition in the 
sense that it does not lead to a trichotomy: there will be unequal Cauchy sequences 
{tn} and {yn} for which neither is less than the other, e.g. 


{vn} = {0,1,1,...}, {yn} = {1,0,0,...}. 
As in the definition of convergence, it is more fruitful to pay attention to what 


a Cauchy sequence is doing eventually. Exploiting this idea we can get a sort of 
trichotomy result. 


LEMMA 16.19. (Cauchy Trichotomy) For a Cauchy sequence {x7} in an ordered 
field F', exactly one of the following holds: 
(i) There is a positive element ¢ € F and N € Z* such that x, > € for alln>N. 
(ii) There is a positive element e € F and N € Z* such that x, < —e foralln > N. 
(itt) tp converges to 0. 


PROOF. It is easy to see that the conditions are mutually exclusive. Let us 
suppose that (iii) does not hold: thus there is « > 0 and a subsequence {zp, } 
such that |z,,| > ¢€ for all k € Z*. By passing to a further subsequence we may 
assume either that z,, > € for all k or x, < —e for all k. Let us suppose that 
the former holds and show that this implies (i): if so, replacing « by —x shows 
that the latter alternative implies (ii). Since {x,,} is Cauchy, there is N € Zt such 
that |tm —%n| < 7 for allm,n > N. Putting these two conditions together we get 
Ly = 5 for alln > N. 


Unfortunately this is not quite the kind of trichotomy which defines a total order 
relation: we have some elements that we regard as positive — case (i) above, some 
elements that we regard as negative — case (ii) above — but for an order relation 
only the zero element should be neither positive nor negative, whereas case (iii) 
above includes the much larger collection of elements converging to zero. 


5for those who know some abstract algebra: what we’ve shown is that (C,+,-) is a commu- 
tative ring. There is a very general algebraic method which, when given a commutative ring, 
will yield a collection of fields defined in terms of that ring. The present construction is indeed 
an instance of this. 
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Lemma 16.19 suggests that if we could somehow “squash down” the subset of 
Cauchy sequences which converge to 0 to a single point, then we would actually get 
a total order relation. This business of “squashing subsets to a point” is formalized 
in mathematics (more so in algebra and topology than the part of mathematics 
we’ve been studying for most of this text!) by an equivalence relation. Rather 
than providing a logically complete but pedagogically useless whirlwind tour of 
equivalence relations, we will simply assume that the reader is familiar with them.® 
Namely, we regard any two Cauchy sequences which converge to 0 as equivalent. 
We are left with the question of when to regard two Cauchy sequences which do 
not converge to zero as equivalent. We could simply “not squash them”, i.e., de- 
clare two such sequences to be equivalent exactly when they are equal. But a 
little exploration shows that this won’t work: we’ll get a total order relation but it 
won’t interact well with the algebraic structure. For instance, consider the Cauchy 
sequences 
{Get = (O01 Tifa} = dT elo sy) 

The difference {x,,} — {yn} converges to 0 so is getting identified with 0. Thus we 
should identify {2,} and {y,} as well. This leads to the following key definitions. 


Let Z be the set of all sequences in F' which converge to 0; convergent sequences 
are Cauchy, so certainly Z C C. For two Cauchy sequences de, be € C, we put 


dew~be — a —-b E Z. 
In words, two Cauchy sequences are equivalent iff their difference converges to zero. 


EXERCISE 16.21. (For those who know some ring theory.) Show that Z is a 
maximal ideal in the commutative ring C. Why is this an exciting sign that we’re 
on the right track? 


EXERCISE 16.22. Let {tn} be a Cauchy sequence in F’, and let {xn,} be any 
subsequence. Show that {x,} ~ {%n,}. 


Now we define R as C/ ~, that is, the set of equivalence classes of Cauchy sequences. 
This will be the underlying set of our sequential completion. It remains to endow it 
with all the rest of the structure. The idea here is that when we pass to a quotient 
by an equivalence relation we can try to simply carry over the structure we already 
had, but at every step we must check that the operations are well-defined, mean- 
ing they are independent of the chosen equivalence class. At no point are these 
verifications difficult, but we admit they can be somewhat tedious. 


Let us check the addition and multiplication induced well-defined operations on 
the set R of equivalence classes. This means: if we have four Cauchy sequences 
Ge, ve, Ce, de and Ae ~ Co, be ~ de, then 


det be ~Cotde, Gebe ~ Code. 
All right: since ag ~ c, and b, ~ dy, de — ce — 0 and b, — d, — 0, so 
(aa+be —(c +d.) = (de Ce) t (be de) +0 t 0 =0, 


6 at UGA they are covered in the “transitional” Math 3200 course. The reader who has made 
it through most of this text will have no problem learning this concept. 
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SO de +b. ~ ce +d,. Similarly, 
Gebe = Code = (ae = Co)be se (be = de)Ce, 


and this converges to 0 because de — Ce, be — de — 0 and be, ce are bounded. Thus 
we have equipped our set ® with two binary operations + and -. 


PROPOSITION 16.20. (R,+,-) is a field. 


ProoF. Most of these axioms are completely straightforward (but, yes, some- 
what tedious) to verify and are left to the reader as exercises. Let us single out: 
(P3) The additive identity is [0.], the class of the constant sequence 0. 

(P7) The multiplicative identity is [1,], the class of the constant sequence 1. 

(P8) Suppose that x € R \ {[0.]}, and let xz be any Cauchy sequence representing 
x. Then we must have x, # 0 for all sufficiently large n: indeed, otherwise we 
would have 0. = (0,0,0,...) as a subsequence, and if a subsequence of a Cauchy 
sequence converges to 0, then the Cauchy sequence itself converges to 0, contradic- 
tion. Suppose z,, # 0 for alln > N. Then define y. by y, = 0 forall l <n< WN 
(or whatever you want: it doesn’t matter) and y, = mn forn > N. Then tnyn =1 
for alln > N, so eye differs from 1, by a sequence which is convergent to zero: 
[te][Ye = [1-] = 1, so y = [ye] is the multiplicative inverse of x = [2]. 


We now equip R with an ordering. For a,b. € C, we put [ae] > [be] if there is a 
positive element ¢ in F such that a, — b, > € for all sufficiently large n. We claim 
that this is well-defined independent of the representatives a, and b, chosen. Indeed, 
if 2, and y. converge to zero, then for sufficiently large n we have |%m — Yn| < § 
and then 


An + 2p) — (bn + Yn) = (@n — bn) + (An — Yn) = - 


THEOREM 16.21. The strict ordering < on R endows it with the structure of 
an ordered field. 


PROOF. Step 1: The trichotomy property follows from Lemma 16.19. The 
transitive property is easy: if [as] < [be] and [b.] < [ce], then there is N € Zt 
and €1,€2 > 0 such that for all n > N, bh — an > €& and cp — bn > €2, hence 
Cn — On = (Cn — bn) + (bn — Gn) > €1 + €2. Thus < is a strict ordering on R. 

Step 2: If [ae], [be] > 0, then as above there is N € Zt and €1,€2 > 0 such that 
for alln > N, an > €, and b, > €g. Then for alln > N, an + bn > € + €2 and 
Anbn > €1€2. 


PROPOSITION 16.22. For an ordered field F, lett: F > R by sendingx e F 
to the class of the constant sequence (x,x,...). Then vu is a dense homomorphism 
of ordered fields. 


PROOF. Step 1: It is immediate that s is a field homomorphism. Further, if 
x > 0, then (a#,2,...) > 0: indeed, we can take « = x. So u is a homomorphism of 
ordered fields. 
Step 2: Let x = [a,] in R. We claim that u(a,) — x. Indeed, if € = [e,] is a positive 
element of R, then there is e > 0 in F such that ¢, > € for all sufficiently large n. 
Since {a,} is Cauchy in F’, there is N € Z* such that for all m,n>N, 


e 
|an — @m| < rt 


2. THE SEQUENTIAL COMPLETION 351 


Then for sufficiently large m, 


Em — |@n — Im| > € — |an — Gm| = ’ 


Nl] oO 


which shows that for all n > N, |t(an) — z| < €. Step 3: It remains to show that 
t is dense, so let a = [a.] < b = [b.] € R. Let m = “4% and let « = 254; thus the 
interval of width € centered at m has as its endpoints a and b. By Step 2, there is 
a sequence {z,,} ¢€ F' such that .(z,) + m. By definition then, for all sufficiently 


large n we have|z(zn) — m| < €, and thus 1(ap) € (a, 6). 


Finally, we prove Theorem 16.16 in the following explicit form. 


THEOREM 16.23. For any ordered field F, the homomorphism 1: F > R= 
C/Z is a sequential completion. Moreover, if L is a sequentially complete field and 
f:F > L is a cofinal ordered field homomorphism, then there is a unique ordered 
field homomorphism g:R — L such that f=gout. 


PRooF. We already know that 1: F — FR is a dense ordered field homomor- 
phism. It remains to check first that R is sequentially complete and second that 4 
satisfies the “universal property” of Theorem 16.16b). 

Step 1: Let {2,} be a Cauchy sequence in R; we must show that it is conver- 
gent. By Proposition 16.9 it suffices to check this after passing to a subsequence. 
If {zy} has only finitely many distinct points, we have a constant subsequence, 
which is certainly convergent. Otherwise {x,,} has infinitely many distinct points, 
so that after passage to a subsequence we may assume they are all distinct. Since 
by Proposition 16.22 1 is a dense homomorphism, for all n € Z* there is y, € F 
such that y, lies strictly between x, and vp41. It follows that for any n,k € Z*, 
Yn+k lies in any interval containing @,%n41,---;n+r+1- Since {x,} is Cauchy, 
this forces {yp} be to Cauchy, and thus y = [{yn}] € R. We also have tp — yn, > 0, 
80 limp oo Ln = LiMn—+oo Yn = Y- 
Step 2: Let L be a sequentially complete ordered field, and let f: F ~ L bea 
cofinal ordered field homomorphism. We must show that there is a unique ordered 
field homomorphism g: R — L such that f =gov. 
Uniqueness: Let g : R — L be such that f = gov. We will see explicitly what 
g must be on each « € R. Write x = [a,]. Since {a,} is Cauchy in F and f is 
cofinal, f is uniformly continuous, so {f(a,,)} is Cauchy in L, hence convergent, say 
toy € L. Since f is cofinal, is g, hence g is uniformly continuous. Since u(a,) > 2, 
f(@n) = g(u(an)) > g(x). Since f(an) > y and limits are unique, we must have 
g(x) =y= lim flan). 
Existence: We must show that putting g(z) = y as above defines an ordered field 
homomorphism from R to L. If #1 = [an] and x2 = [by], let yr: = limn+c f(an) 
and yo = limn+yoo f(bn). Then an + bp > 1 + @2 and f(an + bn) = f(an) + 
f(bn) > yr + yo, 80 g(#@1 + 2) = yr + y2 = g(a1) + g(a2). Similarly, anbn > x12 
and f(dnbn) = f(an)f(bn) > yry2, 80 g(t1%2) = yiy2 = g(x1)g(x2). Finally, if 
x = [ae] > 0, then there is a positive « € F' such that a, > € for all sufficiently large 
n, SO 


g(x) = lim f(an) > lim f(e) = FQ) >0. 


n—-oo n—-oo 
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